Jul 16 21:13:20.370268 ip-10-0-133-67 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 16 21:13:20.370282 ip-10-0-133-67 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 16 21:13:20.370291 ip-10-0-133-67 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 16 21:13:20.370550 ip-10-0-133-67 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 16 21:13:30.609277 ip-10-0-133-67 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 16 21:13:30.609293 ip-10-0-133-67 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d2a2fe5daecb4cea971c6977f4f2f2ff --
Jul 16 21:15:33.211727 ip-10-0-133-67 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 21:15:34.979467 ip-10-0-133-67 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:15:34.979467 ip-10-0-133-67 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 21:15:34.979467 ip-10-0-133-67 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:15:34.979467 ip-10-0-133-67 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 21:15:34.979467 ip-10-0-133-67 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 21:15:34.981849 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.981760    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 21:15:34.986652 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986632    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:34.986652 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986653    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986657    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986661    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986664    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986667    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986670    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986673    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986676    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986679    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986681    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986685    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986689    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986692    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986695    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986698    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986700    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986704    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986708    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986711    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:34.986723 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986714    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986717    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986720    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986723    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986726    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986729    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986732    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986735    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986738    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986741    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986743    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986746    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986749    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986751    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986754    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986757    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986760    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986763    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986765    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986767    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:34.987154 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986770    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986772    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986776    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986779    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986781    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986783    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986786    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986790    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986794    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986797    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986800    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986803    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986806    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986808    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986812    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986815    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986818    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986820    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986823    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:34.987670 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986826    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986829    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986832    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986835    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986837    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986840    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986843    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986846    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986848    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986852    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986855    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986857    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986860    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986863    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986865    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986868    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986871    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986874    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986877    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:34.988135 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986880    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986882    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986884    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986887    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986890    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986892    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986895    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986897    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986900    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986904    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986906    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986909    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986911    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986914    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986917    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986919    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986922    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986924    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986927    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986929    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:34.988606 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.986932    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987710    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987717    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987720    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987724    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987728    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987731    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987733    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987736    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987739    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987742    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987744    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987747    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987749    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987752    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987756    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987759    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987761    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987764    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:34.989102 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987766    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987769    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987771    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987774    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987777    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987780    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987782    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987785    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987787    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987790    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987792    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987795    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987798    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987802    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987805    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987808    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987810    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987813    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987818    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:34.989705 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987822    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987825    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987828    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987832    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987835    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987838    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987840    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987843    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987845    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987848    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987851    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987853    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987856    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987858    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987860    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987863    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987866    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987868    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987871    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987873    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:34.990203 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987876    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987878    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987881    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987883    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987886    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987888    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987891    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987894    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987897    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987900    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987902    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987905    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987908    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987910    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987913    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987916    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987918    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987920    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987923    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:34.990689 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987926    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987930    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987932    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987935    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987938    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987942    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987945    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987947    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987950    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987953    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987955    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987958    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987961    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987964    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987966    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987969    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987971    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987974    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987976    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987979    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:34.991144 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987982    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987984    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.987987    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988451    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988456    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988460    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988463    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988465    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988468    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988471    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988473    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988476    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988479    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988483    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988486    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988490    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988493    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988496    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988499    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:34.991641 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988501    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988505    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988507    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988510    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988512    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988515    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988518    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988520    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988523    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988526    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988528    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988531    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988533    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988536    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988538    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988541    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988543    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988546    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988548    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988551    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:34.992098 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988554    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988556    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988558    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988561    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988563    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988566    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988569    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988571    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988574    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988577    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988579    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988582    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988584    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988587    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988589    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988591    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988594    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988596    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988599    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988601    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:34.992599 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988604    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988606    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988609    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988611    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988614    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988617    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988622    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988625    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988627    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988632    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988636    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988639    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988642    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988645    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988648    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988650    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988653    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988656    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988658    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:34.993065 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988661    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988664    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988666    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988669    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988672    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988674    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988677    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988679    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988682    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988684    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988687    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988691    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988694    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988697    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988701    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988703    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988706    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988708    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988711    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988715    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:34.993538 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988717    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988720    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988722    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.988725    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988824    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988835    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988843    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988848    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988854    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988860    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988865    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988870    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988874    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988877    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988880    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988884    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988887    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988890    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988893    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988896    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988899    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988902    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988906    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 16 21:15:34.993995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988909    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988912    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988915    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988918    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988922    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988925    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988928    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988932    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988935    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988938    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988941    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988944    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988947    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988952    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988956    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988960    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988963    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988966    2572 flags.go:64] FLAG: --enable-server="true"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988970    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988975    2572 flags.go:64] FLAG: --event-burst="100"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988978    2572 flags.go:64] FLAG: --event-qps="50"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988981    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988984    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988987    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 16 21:15:34.994561 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988991    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988993    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988996    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.988999    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989002    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989005    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989009    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989012    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989015    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989018    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989020    2572 flags.go:64] FLAG: --feature-gates=""
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989024    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989027    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989030    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989033    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989036    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989040    2572 flags.go:64] FLAG: --help="false"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989042    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-133-67.ec2.internal"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989045    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989049    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989052    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989056    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989060    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989064    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 21:15:34.995122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989067    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989071    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989074    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989077    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989080    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989083    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989086    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989089    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989092    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989094    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989097    2572 flags.go:64] FLAG: --lock-file=""
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989100    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989103    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989106    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989113    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989116    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989119    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989122    2572 flags.go:64] FLAG: --logging-format="text"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989125    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989128    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989131    2572 flags.go:64] FLAG: --manifest-url=""
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989134    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989138    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989141    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989146    2572 flags.go:64] FLAG: --max-pods="110"
Jul 16 21:15:34.995782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989149    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989152    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989155    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989158    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989164    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989168    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989171    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989179    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989182    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989185    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989188    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989191    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989197    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989199    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989203    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989206    2572 flags.go:64] FLAG: --port="10250"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989209    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989212    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02266499a6d82d783"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989215    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989218    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989221    2572 flags.go:64] FLAG: --register-node="true"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989224    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989228    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989231    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 16 21:15:34.996407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989233    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989236    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989240    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989243    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989246    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989249    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989252    2572 flags.go:64] FLAG: --runonce="false"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989254    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989257    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989260    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989263    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989266    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989272    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989275    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989280    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989283    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989286    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989289    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989292    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989295    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989298    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989301    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989307    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989310    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989313    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 21:15:34.997002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989317    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989319    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989322    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989325    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989328    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989331    2572 flags.go:64] FLAG: --v="2"
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989335    2572 flags.go:64] FLAG: --version="false"
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989339    2572 flags.go:64] FLAG: --vmodule=""
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989343    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.989347    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989482    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989486    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989489    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989492    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989495    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989498    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989500    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989503    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989505    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989509    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989512    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989516    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:34.997637 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989518    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989521    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989524    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989527    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989529    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989532    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989534    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989537    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989539    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989542    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989545    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989547    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989549    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989552    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989554    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989557    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989559    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989562    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989564    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:34.998372 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989569    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989573    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989576    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989579    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989582    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989584    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989588    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989591    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989593    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989596    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989599    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989602    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989606    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989609    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989611    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989614    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989617    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989620    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989623    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989625    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:34.999104 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989628    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989630    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989633    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989635    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989638    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989641    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989643    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989646    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989648    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989651    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989653    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989656    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989658    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989661    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989663    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989666    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989669    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989673    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989676    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:34.999661 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989679    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989681    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989684    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989688    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989690    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989694    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989697    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989700    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989703    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989705    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989709    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989714    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989717    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989720    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989723    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989726    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989729    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989731    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989733    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:35.000105 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989736    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989739    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989741    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989744    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989747    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989749    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989752    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989755    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989757    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:34.989760    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.990548    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.999489    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:34.999511    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.000083    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.000094    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 21:15:35.000707 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000155    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000161    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000164    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000167    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000170    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000173    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000176    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000179    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000181    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000184    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000187    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000190    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000194    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000198    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000201    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000205    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000208    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000210    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000213    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:35.001082 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000216    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000219    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000221    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000224    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000226    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000229    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000231    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000235    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000237    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000240    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000244    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000247    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000250    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000254    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000258    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000261    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000263    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000266    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000269    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:35.001536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000273    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000276    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000279    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000282    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000285    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000288    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000291    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000294    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000298    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000301    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000304    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000307    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000310    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000313    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000315    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000318    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000321    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000324    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000326    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000329    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:35.001992 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000332    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000343    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000346    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000366    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000371    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000374    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000377    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000380    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000383    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000385    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000388    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000391    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000393    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000396    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000399    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000402    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000404    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000407    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000409    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000412    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:35.002497 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000415    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000419    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000421    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000424    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000427    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000429    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000432    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000435    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000437    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000440    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000443    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000446    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000448    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000451    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000454    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000456    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000459    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000461    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000464    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000467    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:35.002963 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000469    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.000475    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000589    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000593    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000597    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000600    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000602    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000607    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000610    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000613    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000617    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000619    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000622    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000625    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 21:15:35.003448 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000628    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000631    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000634    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000636    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000638    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000641    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000644    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000647    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000649    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000652    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000655    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000657    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000659    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000662    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000664    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000667    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000669    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000673    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000675    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000678    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 21:15:35.003800 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000680    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000683    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000685    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000688    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000690    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000693    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000696    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000698    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000701    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000703    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000706    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000709    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000712    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000715    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000718    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000721    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000723    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000726    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000728    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 21:15:35.004267 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000731    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000733    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000736    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000738    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000742    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000746    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000749    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000751    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000754    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000757    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000760    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000762    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000765    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000768    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000770    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000773    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000775    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000778    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000781    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000783    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 21:15:35.004748 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000786    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000792    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000795    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000798    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000800    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000802    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000805    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000809    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000813    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000816    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000819    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000821    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000824    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000826    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000829    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000831    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000834    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000836    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000838    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000841    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 21:15:35.005224 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000844    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000846    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000849    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000851    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000854    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000856    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000859    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:35.000861    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.000866    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.000988    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 21:15:35.005774 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.004974    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 21:15:35.006334 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.006319    2572 server.go:1019] "Starting client certificate rotation"
Jul 16 21:15:35.006443 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.006424    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 21:15:35.006473 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.006461    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 21:15:35.008955 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.008940    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 21:15:35.011720 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.011695    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 21:15:35.021165 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.021106    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 16 21:15:35.025274 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.025254    2572 log.go:25] "Validated CRI v1 image API"
Jul 16 21:15:35.026077 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.026061    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 21:15:35.028132 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.028108    2572 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a4dc1fae-0a6a-4021-b346-4a680b4f7095:/dev/nvme0n1p3 a94260ee-eae0-4523-b89c-52d4658b9bde:/dev/nvme0n1p4]
Jul 16 21:15:35.028213 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.028132    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 21:15:35.034760 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.034640    2572 manager.go:216] Machine: {Timestamp:2026-07-16 21:15:35.033320763 +0000 UTC m=+0.544480500 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101292 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bf3d7ef5bf6a6e021b2858fa3e09c SystemUUID:ec2bf3d7-ef5b-f6a6-e021-b2858fa3e09c BootID:d2a2fe5d-aecb-4cea-971c-6977f4f2f2ff Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:87:48:4b:43:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:87:48:4b:43:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:6b:9b:57:b8:a4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 21:15:35.034760 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.034752    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 21:15:35.034906 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.034836    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 21:15:35.037466 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.037439    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 21:15:35.037608 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.037467    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-67.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 21:15:35.038237 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.038228    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 21:15:35.038267 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.038240    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 21:15:35.038267 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.038247    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 21:15:35.039500 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.039489    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 21:15:35.041154 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.041144    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 21:15:35.044856 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.044842    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 21:15:35.044968 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.044958    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 21:15:35.045009 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.044987    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 21:15:35.045009 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.044997    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 21:15:35.045009 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.045008    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 21:15:35.045125 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.045017    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 16 21:15:35.045125 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.045026    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 21:15:35.045591 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.045574    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 21:15:35.045684 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.045612    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 21:15:35.047046 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.047022    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 21:15:35.048764 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.048743    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 21:15:35.050457 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.050441    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 21:15:35.050528 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.050471    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 21:15:35.051234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051223    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 21:15:35.051265 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051239    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 21:15:35.051265 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051245    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 21:15:35.051265 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051254    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 21:15:35.051265 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051264    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 21:15:35.051400 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051270    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 21:15:35.051400 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051279    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 21:15:35.051400 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051287    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 21:15:35.051400 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051296    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 21:15:35.051400 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051302    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 21:15:35.051400 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051311    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 21:15:35.051400 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.051317    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 21:15:35.052744 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.052732    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 21:15:35.052744 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.052744    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 21:15:35.055841 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.055826    2572 server.go:1268] "Started kubelet"
Jul 16 21:15:35.055929 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.055900    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 21:15:35.055989 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.055936    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 21:15:35.056033 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.056015    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 21:15:35.056790 ip-10-0-133-67 systemd[1]: Started Kubernetes Kubelet.
Jul 16 21:15:35.057431 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.057390    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 21:15:35.057753 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.057607    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 21:15:35.061093 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.061076    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 21:15:35.061931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.061911    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-67.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 21:15:35.061931 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.061915    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-67.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 21:15:35.062932 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.061913    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-67.ec2.internal.18c2e1ace34649ee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-67.ec2.internal,UID:ip-10-0-133-67.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-67.ec2.internal,},FirstTimestamp:2026-07-16 21:15:35.055796718 +0000 UTC m=+0.566956443,LastTimestamp:2026-07-16 21:15:35.055796718 +0000 UTC m=+0.566956443,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-67.ec2.internal,}"
Jul 16 21:15:35.065553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.065533    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 21:15:35.065664 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.065533    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 21:15:35.065727 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.065684    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.066199 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066180    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 21:15:35.066199 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066200    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 21:15:35.066325 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066198    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 21:15:35.066424 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066412    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 21:15:35.066476 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066427    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 21:15:35.066520 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.066478    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 21:15:35.066695 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066678    2572 factory.go:153] Registering CRI-O factory
Jul 16 21:15:35.066764 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066699    2572 factory.go:223] Registration of the crio container factory successfully
Jul 16 21:15:35.066764 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066753    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 21:15:35.066764 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066764    2572 factory.go:55] Registering systemd factory
Jul 16 21:15:35.066928 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066772    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 16 21:15:35.066928 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066794    2572 factory.go:103] Registering Raw factory
Jul 16 21:15:35.066928 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.066807    2572 manager.go:1196] Started watching for new ooms in manager
Jul 16 21:15:35.067196 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.067181    2572 manager.go:349] Starting recovery of all containers
Jul 16 21:15:35.069399 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.069376    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 21:15:35.069500 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.069385    2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-67.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 21:15:35.077026 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.076990    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q9lt4"
Jul 16 21:15:35.079616 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.079594    2572 manager.go:354] Recovery completed
Jul 16 21:15:35.084119 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.084104    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:35.086005 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.085990    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:35.086066 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.086021    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:35.086066 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.086034    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:35.086637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.086622    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 21:15:35.086707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.086640    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 21:15:35.086707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.086626    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:35.086707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.086658    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 21:15:35.087360 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.087331    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:35.087458 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.087376    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:35.087458 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.087392    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:35.087968 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.087954    2572 policy_none.go:49] "None policy: Start"
Jul 16 21:15:35.088026 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.087975    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 21:15:35.088026 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.087989    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 21:15:35.089331 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.089318    2572 policy_none.go:47] "Start"
Jul 16 21:15:35.094396 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.094307    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-67.ec2.internal.18c2e1ace5134091  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-67.ec2.internal,UID:ip-10-0-133-67.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-67.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-67.ec2.internal,},FirstTimestamp:2026-07-16 21:15:35.086006417 +0000 UTC m=+0.597166144,LastTimestamp:2026-07-16 21:15:35.086006417 +0000 UTC m=+0.597166144,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-67.ec2.internal,}"
Jul 16 21:15:35.096552 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.096535    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q9lt4"
Jul 16 21:15:35.123868 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.123696    2572 manager.go:337] "Starting Device Plugin manager"
Jul 16 21:15:35.123972 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.123895    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 21:15:35.123972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.123907    2572 server.go:87] "Starting device plugin registration server"
Jul 16 21:15:35.124123 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.124113    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 21:15:35.124162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.124124    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 21:15:35.124262 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.124242    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 21:15:35.124369 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.124328    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 21:15:35.124474 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.124447    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 21:15:35.124886 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.124867    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 21:15:35.124957 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.124923    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.224754 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.224723    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:35.226156 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.225783    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:35.226156 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.225817    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:35.226156 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.225833    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:35.226156 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.225861    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.241798 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.241764    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 21:15:35.242067 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.242043    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.242183 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.242074    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-67.ec2.internal\": node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.243029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.243009    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 21:15:35.243116 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.243045    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 21:15:35.243116 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.243065    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 21:15:35.243116 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.243099    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 21:15:35.250165 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.250145    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:35.343841 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.343758    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal"]
Jul 16 21:15:35.343981 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.343857    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:35.344939 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.344923    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:35.345020 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.344953    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:35.345020 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.344968    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:35.346344 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.346331    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:35.346463 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.346450    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.346502 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.346483    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:35.347083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.347069    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:35.347148 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.347097    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:35.347148 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.347107    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:35.347148 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.347072    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:35.347237 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.347157    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:35.347237 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.347166    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:35.348518 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.348505    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.348566 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.348528    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:35.349204 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.349189    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:35.349272 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.349216    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:35.349272 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.349227    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:35.363438 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.363416    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.367560 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.367541    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a050b69ff95c7f4d535f7563e4b39735-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal\" (UID: \"a050b69ff95c7f4d535f7563e4b39735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.367637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.367571    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5112854b758de5dd94996dccba2987eb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-67.ec2.internal\" (UID: \"5112854b758de5dd94996dccba2987eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.367637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.367587    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a050b69ff95c7f4d535f7563e4b39735-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal\" (UID: \"a050b69ff95c7f4d535f7563e4b39735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.373297 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.373275    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-67.ec2.internal\" not found" node="ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.377738 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.377719    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-67.ec2.internal\" not found" node="ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.464046 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.464009    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.468402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.468379    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a050b69ff95c7f4d535f7563e4b39735-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal\" (UID: \"a050b69ff95c7f4d535f7563e4b39735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.468539 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.468418    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5112854b758de5dd94996dccba2987eb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-67.ec2.internal\" (UID: \"5112854b758de5dd94996dccba2987eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.468539 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.468441    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a050b69ff95c7f4d535f7563e4b39735-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal\" (UID: \"a050b69ff95c7f4d535f7563e4b39735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.468539 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.468481    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a050b69ff95c7f4d535f7563e4b39735-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal\" (UID: \"a050b69ff95c7f4d535f7563e4b39735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.468539 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.468494    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a050b69ff95c7f4d535f7563e4b39735-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal\" (UID: \"a050b69ff95c7f4d535f7563e4b39735\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.468539 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.468502    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5112854b758de5dd94996dccba2987eb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-67.ec2.internal\" (UID: \"5112854b758de5dd94996dccba2987eb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.565176 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.565139    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.665982 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.665913    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.675693 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.675671    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.683157 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:35.683132    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:35.766542 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.766508    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.866871 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.866837    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:35.967389 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:35.967284    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:36.005624 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.005600    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 21:15:36.006066 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.005742    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 21:15:36.067470 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:36.067443    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:36.101805 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.101750    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 21:10:35 +0000 UTC" deadline="2028-02-22 05:43:25.306586733 +0000 UTC"
Jul 16 21:15:36.101805 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.101800    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14048h27m49.2047904s"
Jul 16 21:15:36.167761 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:36.167729    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:36.224997 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.224959    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 21:15:36.226396 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.226371    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 21:15:36.226499 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.226413    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 21:15:36.226499 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.226423    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeHasSufficientPID"
Jul 16 21:15:36.226905 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:36.226883    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda050b69ff95c7f4d535f7563e4b39735.slice/crio-3dfdb050266fd3a2a92a3e73dc3d0d571f976335f09a9ed144763ff6ba1f0c13 WatchSource:0}: Error finding container 3dfdb050266fd3a2a92a3e73dc3d0d571f976335f09a9ed144763ff6ba1f0c13: Status 404 returned error can't find the container with id 3dfdb050266fd3a2a92a3e73dc3d0d571f976335f09a9ed144763ff6ba1f0c13
Jul 16 21:15:36.227380 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:36.227340    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5112854b758de5dd94996dccba2987eb.slice/crio-49ef965bb6030259f4cd1c826907560677eb84c4a862b58f7c8b15244c0fb2b4 WatchSource:0}: Error finding container 49ef965bb6030259f4cd1c826907560677eb84c4a862b58f7c8b15244c0fb2b4: Status 404 returned error can't find the container with id 49ef965bb6030259f4cd1c826907560677eb84c4a862b58f7c8b15244c0fb2b4
Jul 16 21:15:36.231498 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.231479    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:15:36.248430 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.248380    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal" event={"ID":"a050b69ff95c7f4d535f7563e4b39735","Type":"ContainerStarted","Data":"3dfdb050266fd3a2a92a3e73dc3d0d571f976335f09a9ed144763ff6ba1f0c13"}
Jul 16 21:15:36.249381 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.249344    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal" event={"ID":"5112854b758de5dd94996dccba2987eb","Type":"ContainerStarted","Data":"49ef965bb6030259f4cd1c826907560677eb84c4a862b58f7c8b15244c0fb2b4"}
Jul 16 21:15:36.268370 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:36.268316    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:36.369362 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:36.369311    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:36.386117 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.386093    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:36.470219 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:36.470174    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:36.571072 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:36.570987    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-67.ec2.internal\" not found"
Jul 16 21:15:36.649223 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.649201    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:36.653392 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.653375    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:36.665940 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.665915    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:36.697138 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.697109    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 21:15:36.698179 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.698164    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal"
Jul 16 21:15:36.730383 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:36.730336    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 21:15:37.047141 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.047050    2572 apiserver.go:52] "Watching apiserver"
Jul 16 21:15:37.067109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.067073    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 21:15:37.068420 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.068393    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 21:15:37.068858 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.068837    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal","openshift-multus/network-metrics-daemon-2wwbb","openshift-network-operator/iptables-alerter-zwmb7","kube-system/konnectivity-agent-mnn2h","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp","openshift-dns/node-resolver-zchn6","openshift-image-registry/node-ca-qc6g5","openshift-multus/multus-9fw8z","openshift-multus/multus-additional-cni-plugins-kn78t","openshift-network-diagnostics/network-check-target-ddb52","openshift-ovn-kubernetes/ovnkube-node-vstcz","kube-system/global-pull-secret-syncer-zjlgl","kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal","openshift-cluster-node-tuning-operator/tuned-pnzkq"]
Jul 16 21:15:37.070471 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.070445    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.072967 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.072755    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.074679 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.074661    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.075798 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.075780    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:37.076416 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.076395    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 21:15:37.076513 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.076438    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 21:15:37.076513 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.076480    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rk2l4\""
Jul 16 21:15:37.076608 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.076399    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 21:15:37.077121 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.077102    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.080455 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.080434    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.082192 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.082171    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.084058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.084039    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.084261 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.084149    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gqv4l\""
Jul 16 21:15:37.085964 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.085943    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:37.086060 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.086025    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:37.089526 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.087757    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:37.089526 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.087858    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:37.091010 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.090961    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.092408 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.092297    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.092408 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.092399    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:37.094059 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.094012    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 21:15:37.094523 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.094303    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 21:15:37.094523 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.094388    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 21:15:37.094523 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.094400    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 21:15:37.094523 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.094414    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 21:15:37.094523 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.094477    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 21:15:37.094806 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.094547    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:15:37.094806 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.094574    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 21:15:37.097832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.097806    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 21:15:37.097924 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.097853    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cfdgr\""
Jul 16 21:15:37.099530 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.099488    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 21:15:37.103621 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.103278    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 21:15:37.103621 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.103535    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wzqz2\""
Jul 16 21:15:37.105392 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.104696    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 21:15:37.105392 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.104781    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 21:15:37.105392 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.104911    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bngwc\""
Jul 16 21:15:37.105392 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.105165    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 21:15:37.105392 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.105373    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 21:15:37.105964 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.104701    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 21:15:37.105964 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.105702    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 21:15:37.105964 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.105823    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 21:15:37.105964 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.105883    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 21:15:37.106214 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.106051    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-khpmk\""
Jul 16 21:15:37.106214 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.106056    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q4qlx\""
Jul 16 21:15:37.106314 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.106240    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 21:15:37.106782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.106427    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-glk7c\""
Jul 16 21:15:37.106782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.106463    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:15:37.106782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.106608    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mjjgg\""
Jul 16 21:15:37.106782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.106637    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 21:15:37.106782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.106703    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 21:15:37.131578 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.131555    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 21:15:37.139308 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.139283    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 21:15:37.168222 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.168201    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 21:15:37.168456 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.168438    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-crsxc"
Jul 16 21:15:37.177684 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-device-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.177819 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177691    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-run-netns\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.177819 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177710    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-os-release\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.177819 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177735    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-cni-binary-copy\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.177819 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177757    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-multus-certs\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.177819 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177781    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djz5l\" (UniqueName: \"kubernetes.io/projected/a5c76402-3101-42e3-9af0-72d87c627513-kube-api-access-djz5l\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.177819 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177806    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cnibin\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177831    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177855    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177879    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-etc-selinux\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177905    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-daemon-config\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177947    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovnkube-script-lib\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177973    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-kubernetes\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.177993    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-tuned\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178015    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr2kv\" (UniqueName: \"kubernetes.io/projected/9229aed3-7c79-40f2-8886-21024b1b8ec9-kube-api-access-kr2kv\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178038    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-cni-bin\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.178094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178064    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-etc-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178109    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5c76402-3101-42e3-9af0-72d87c627513-iptables-alerter-script\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178157    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178186    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtmmn\" (UniqueName: \"kubernetes.io/projected/c865333f-a4ab-4780-80ce-7463b73a6acc-kube-api-access-xtmmn\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178213    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9229aed3-7c79-40f2-8886-21024b1b8ec9-tmp\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178238    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-cni-multus\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178264    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-conf-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178284    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-slash\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178300    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovnkube-config\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178316    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8572761-a70c-4a5c-a04a-2ffc3ccec888-host\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178330    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-node-log\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178363    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-log-socket\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178377    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5c76402-3101-42e3-9af0-72d87c627513-host-slash\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178391    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7b58d65-9c6a-43c1-b913-981279a5ace0-konnectivity-ca\") pod \"konnectivity-agent-mnn2h\" (UID: \"d7b58d65-9c6a-43c1-b913-981279a5ace0\") " pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178409    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-system-cni-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178430    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-systemd-units\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.178558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178466    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-run-ovn-kubernetes\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178512    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-registration-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178561    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-hosts-file\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178599    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-modprobe-d\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178624    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-run\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178653    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-lib-modules\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178675    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-cnibin\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178697    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-kubelet\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178732    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-env-overrides\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178760    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7b58d65-9c6a-43c1-b913-981279a5ace0-agent-certs\") pod \"konnectivity-agent-mnn2h\" (UID: \"d7b58d65-9c6a-43c1-b913-981279a5ace0\") " pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178783    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-socket-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178808    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysconfig\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178832    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysctl-d\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178854    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-cni-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-var-lib-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178903    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178929    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s78l\" (UniqueName: \"kubernetes.io/projected/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-kube-api-access-5s78l\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.179321 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178965    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-sys\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.178988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-host\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179012    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-socket-dir-parent\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179036    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/61292e97-d6af-40aa-9f6b-c944ebe621b3-kubelet-config\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179059    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179081    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvdxt\" (UniqueName: \"kubernetes.io/projected/74cb201b-9a56-4551-8a30-dd8dfc1f2579-kube-api-access-qvdxt\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179108    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjrnt\" (UniqueName: \"kubernetes.io/projected/69d0fe7a-f6b7-43c4-a258-31bf4790975e-kube-api-access-fjrnt\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179132    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179620    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-sys-fs\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179647    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-k8s-cni-cncf-io\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179672    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-netns\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179701    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvrzn\" (UniqueName: \"kubernetes.io/projected/885f0bbb-17f9-4438-b179-2d716e4cc295-kube-api-access-pvrzn\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179723    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8572761-a70c-4a5c-a04a-2ffc3ccec888-serviceca\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179747    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-tmp-dir\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179769    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysctl-conf\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179791    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-etc-kubernetes\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.180162 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179814    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bvrn\" (UniqueName: \"kubernetes.io/projected/d8572761-a70c-4a5c-a04a-2ffc3ccec888-kube-api-access-7bvrn\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179837    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-kubelet\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179858    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-ovn\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179898    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-cni-netd\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179922    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-systemd\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179958    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns9kd\" (UniqueName: \"kubernetes.io/projected/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-kube-api-access-ns9kd\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.179992    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovn-node-metrics-cert\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180016    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-system-cni-dir\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180041    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-os-release\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180062    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-hostroot\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180132    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180156    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-var-lib-kubelet\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180178    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/61292e97-d6af-40aa-9f6b-c944ebe621b3-dbus\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180202    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-systemd\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180225    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-cni-bin\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180257    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.180872 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180281    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.181576 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.180304    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.191560 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.191536    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-crsxc"
Jul 16 21:15:37.280915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.280877    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-sys\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.280915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.280918    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-host\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.280939    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-socket-dir-parent\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.280961    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/61292e97-d6af-40aa-9f6b-c944ebe621b3-kubelet-config\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.280998    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281023    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvdxt\" (UniqueName: \"kubernetes.io/projected/74cb201b-9a56-4551-8a30-dd8dfc1f2579-kube-api-access-qvdxt\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281022    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-host\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281047    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjrnt\" (UniqueName: \"kubernetes.io/projected/69d0fe7a-f6b7-43c4-a258-31bf4790975e-kube-api-access-fjrnt\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281069    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281084    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-sys-fs\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281099    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-k8s-cni-cncf-io\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281121    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-netns\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.281149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281143    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvrzn\" (UniqueName: \"kubernetes.io/projected/885f0bbb-17f9-4438-b179-2d716e4cc295-kube-api-access-pvrzn\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281164    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8572761-a70c-4a5c-a04a-2ffc3ccec888-serviceca\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281185    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-tmp-dir\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281188    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-socket-dir-parent\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.281199    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.281286    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs podName:885f0bbb-17f9-4438-b179-2d716e4cc295 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:37.781247514 +0000 UTC m=+3.292407225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs") pod "network-metrics-daemon-2wwbb" (UID: "885f0bbb-17f9-4438-b179-2d716e4cc295") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281329    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysctl-conf\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281068    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/61292e97-d6af-40aa-9f6b-c944ebe621b3-kubelet-config\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281022    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-sys\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.281408    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281413    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-sys-fs\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281486    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-k8s-cni-cncf-io\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysctl-conf\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281519    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-netns\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.281538    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret podName:61292e97-d6af-40aa-9f6b-c944ebe621b3 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:37.781496804 +0000 UTC m=+3.292656525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret") pod "global-pull-secret-syncer-zjlgl" (UID: "61292e97-d6af-40aa-9f6b-c944ebe621b3") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-etc-kubernetes\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281638    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bvrn\" (UniqueName: \"kubernetes.io/projected/d8572761-a70c-4a5c-a04a-2ffc3ccec888-kube-api-access-7bvrn\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.281762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281673    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-kubelet\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281684    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-etc-kubernetes\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281698    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-ovn\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281743    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-ovn\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281770    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-tmp-dir\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281802    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-kubelet\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281845    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-cni-netd\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281877    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-systemd\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281905    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns9kd\" (UniqueName: \"kubernetes.io/projected/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-kube-api-access-ns9kd\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281932    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovn-node-metrics-cert\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281961    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-system-cni-dir\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281972    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-cni-netd\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281974    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-systemd\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.281989    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-os-release\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282010    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-hostroot\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282032    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8572761-a70c-4a5c-a04a-2ffc3ccec888-serviceca\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282051    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-hostroot\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282058    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-system-cni-dir\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.282553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282114    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-os-release\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282167    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282205    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-var-lib-kubelet\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282239    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/61292e97-d6af-40aa-9f6b-c944ebe621b3-dbus\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282257    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-var-lib-kubelet\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282266    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-systemd\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282245    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282291    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-cni-bin\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282318    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282327    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-systemd\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282332    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282341    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282380    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282387    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-cni-bin\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282397    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/61292e97-d6af-40aa-9f6b-c944ebe621b3-dbus\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282429    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-device-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282447    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.283323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282462    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-run-netns\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282498    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-device-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282517    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-os-release\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282541    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-run-netns\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282545    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-cni-binary-copy\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282574    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-multus-certs\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282598    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djz5l\" (UniqueName: \"kubernetes.io/projected/a5c76402-3101-42e3-9af0-72d87c627513-kube-api-access-djz5l\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282549    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282625    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cnibin\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282638    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-run-multus-certs\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282599    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-os-release\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282651    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282679    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282705    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-etc-selinux\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282729    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-daemon-config\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282751    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovnkube-script-lib\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282776    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-kubernetes\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.284128 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282799    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-tuned\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282822    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr2kv\" (UniqueName: \"kubernetes.io/projected/9229aed3-7c79-40f2-8886-21024b1b8ec9-kube-api-access-kr2kv\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282844    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-cni-bin\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282868    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-etc-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282893    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5c76402-3101-42e3-9af0-72d87c627513-iptables-alerter-script\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282900    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cni-binary-copy\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282893    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-etc-selinux\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282933    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cnibin\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282968    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.282996    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtmmn\" (UniqueName: \"kubernetes.io/projected/c865333f-a4ab-4780-80ce-7463b73a6acc-kube-api-access-xtmmn\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.283021    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9229aed3-7c79-40f2-8886-21024b1b8ec9-tmp\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.283045    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-cni-multus\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.283058    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-cni-binary-copy\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.283070    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-conf-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.283792    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-daemon-config\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.283897    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-cni-multus\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.283965    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-cni-bin\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.284948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.284014    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-etc-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.284138    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovnkube-script-lib\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.284570    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5c76402-3101-42e3-9af0-72d87c627513-iptables-alerter-script\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.284960    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.284973    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-kubernetes\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285058    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-slash\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285097    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovnkube-config\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285129    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8572761-a70c-4a5c-a04a-2ffc3ccec888-host\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285160    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-node-log\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285193    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-log-socket\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285217    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5c76402-3101-42e3-9af0-72d87c627513-host-slash\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285269    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7b58d65-9c6a-43c1-b913-981279a5ace0-konnectivity-ca\") pod \"konnectivity-agent-mnn2h\" (UID: \"d7b58d65-9c6a-43c1-b913-981279a5ace0\") " pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285319    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-system-cni-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285384    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-node-log\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285460    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5c76402-3101-42e3-9af0-72d87c627513-host-slash\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285529    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-systemd-units\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285638    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-run-ovn-kubernetes\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285686    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-registration-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285694    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/69d0fe7a-f6b7-43c4-a258-31bf4790975e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285738    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-hosts-file\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285773    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-modprobe-d\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285835    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-run\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285870    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-lib-modules\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285900    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-cnibin\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285933    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-kubelet\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.285965    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-env-overrides\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286008    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7b58d65-9c6a-43c1-b913-981279a5ace0-agent-certs\") pod \"konnectivity-agent-mnn2h\" (UID: \"d7b58d65-9c6a-43c1-b913-981279a5ace0\") " pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286012    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovn-node-metrics-cert\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286043    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-socket-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286103    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-conf-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286154    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysconfig\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286177    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-systemd-units\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286214    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysctl-d\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286371    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-run\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286442    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-modprobe-d\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.287870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286532    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-system-cni-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286723    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9229aed3-7c79-40f2-8886-21024b1b8ec9-tmp\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286811    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysconfig\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286847    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8572761-a70c-4a5c-a04a-2ffc3ccec888-host\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286879    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-ovnkube-config\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286925    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-socket-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286974    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-run-ovn-kubernetes\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.286986    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-cni-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287027    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-var-lib-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287044    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-multus-cni-dir\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287058    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287089    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s78l\" (UniqueName: \"kubernetes.io/projected/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-kube-api-access-5s78l\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287137    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-sysctl-d\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287183    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-cnibin\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287215    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/74cb201b-9a56-4551-8a30-dd8dfc1f2579-env-overrides\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287235    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-var-lib-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287293    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-run-openvswitch\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287299    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-host-slash\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.288672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287378    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-hosts-file\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.289534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287466    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c865333f-a4ab-4780-80ce-7463b73a6acc-registration-dir\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.289534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287480    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9229aed3-7c79-40f2-8886-21024b1b8ec9-lib-modules\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.289534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287537    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-host-var-lib-kubelet\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.289534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287587    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/74cb201b-9a56-4551-8a30-dd8dfc1f2579-log-socket\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.289534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.287812    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7b58d65-9c6a-43c1-b913-981279a5ace0-konnectivity-ca\") pod \"konnectivity-agent-mnn2h\" (UID: \"d7b58d65-9c6a-43c1-b913-981279a5ace0\") " pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:37.289534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.288041    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9229aed3-7c79-40f2-8886-21024b1b8ec9-etc-tuned\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.291101 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.291063    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7b58d65-9c6a-43c1-b913-981279a5ace0-agent-certs\") pod \"konnectivity-agent-mnn2h\" (UID: \"d7b58d65-9c6a-43c1-b913-981279a5ace0\") " pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:37.339514 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.339057    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:37.339514 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.339093    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:37.339514 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.339114    2572 projected.go:196] Error preparing data for projected volume kube-api-access-tm58n for pod openshift-network-diagnostics/network-check-target-ddb52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:37.339514 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.339195    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n podName:cc75c925-0a50-440b-8def-b991e61e560e nodeName:}" failed. No retries permitted until 2026-07-16 21:15:37.839172744 +0000 UTC m=+3.350332479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tm58n" (UniqueName: "kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n") pod "network-check-target-ddb52" (UID: "cc75c925-0a50-440b-8def-b991e61e560e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:37.341635 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.341222    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjrnt\" (UniqueName: \"kubernetes.io/projected/69d0fe7a-f6b7-43c4-a258-31bf4790975e-kube-api-access-fjrnt\") pod \"multus-additional-cni-plugins-kn78t\" (UID: \"69d0fe7a-f6b7-43c4-a258-31bf4790975e\") " pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.341979 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.341952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvdxt\" (UniqueName: \"kubernetes.io/projected/74cb201b-9a56-4551-8a30-dd8dfc1f2579-kube-api-access-qvdxt\") pod \"ovnkube-node-vstcz\" (UID: \"74cb201b-9a56-4551-8a30-dd8dfc1f2579\") " pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.342081 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.342007    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns9kd\" (UniqueName: \"kubernetes.io/projected/cb4715f5-c6d4-46e5-9918-9e3f3f1cb179-kube-api-access-ns9kd\") pod \"multus-9fw8z\" (UID: \"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179\") " pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.342500 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.342450    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtmmn\" (UniqueName: \"kubernetes.io/projected/c865333f-a4ab-4780-80ce-7463b73a6acc-kube-api-access-xtmmn\") pod \"aws-ebs-csi-driver-node-zd9zp\" (UID: \"c865333f-a4ab-4780-80ce-7463b73a6acc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.342814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.342790    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s78l\" (UniqueName: \"kubernetes.io/projected/6e6e6771-9ce2-4db7-9d34-e9bfba359c4a-kube-api-access-5s78l\") pod \"node-resolver-zchn6\" (UID: \"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a\") " pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.344044 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.344021    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr2kv\" (UniqueName: \"kubernetes.io/projected/9229aed3-7c79-40f2-8886-21024b1b8ec9-kube-api-access-kr2kv\") pod \"tuned-pnzkq\" (UID: \"9229aed3-7c79-40f2-8886-21024b1b8ec9\") " pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.355015 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.354986    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djz5l\" (UniqueName: \"kubernetes.io/projected/a5c76402-3101-42e3-9af0-72d87c627513-kube-api-access-djz5l\") pod \"iptables-alerter-zwmb7\" (UID: \"a5c76402-3101-42e3-9af0-72d87c627513\") " pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.392222 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.392186    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:15:37.394336 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.394296    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvrzn\" (UniqueName: \"kubernetes.io/projected/885f0bbb-17f9-4438-b179-2d716e4cc295-kube-api-access-pvrzn\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:37.402461 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.402431    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zwmb7"
Jul 16 21:15:37.405108 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.405058    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bvrn\" (UniqueName: \"kubernetes.io/projected/d8572761-a70c-4a5c-a04a-2ffc3ccec888-kube-api-access-7bvrn\") pod \"node-ca-qc6g5\" (UID: \"d8572761-a70c-4a5c-a04a-2ffc3ccec888\") " pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.416942 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.416914    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:37.425885 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.425861    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp"
Jul 16 21:15:37.434841 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.434816    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zchn6"
Jul 16 21:15:37.441915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.441749    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kn78t"
Jul 16 21:15:37.453217 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.453153    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pnzkq"
Jul 16 21:15:37.458220 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.458193    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9fw8z"
Jul 16 21:15:37.518340 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.518305    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 21:15:37.683344 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.683272    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qc6g5"
Jul 16 21:15:37.790881 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.790847    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:37.791058 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.790958    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:37.791058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.790980    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:37.791058 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.791027    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret podName:61292e97-d6af-40aa-9f6b-c944ebe621b3 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:38.791006289 +0000 UTC m=+4.302166015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret") pod "global-pull-secret-syncer-zjlgl" (UID: "61292e97-d6af-40aa-9f6b-c944ebe621b3") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:37.791225 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.791085    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:37.791225 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.791134    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs podName:885f0bbb-17f9-4438-b179-2d716e4cc295 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:38.791117958 +0000 UTC m=+4.302277685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs") pod "network-metrics-daemon-2wwbb" (UID: "885f0bbb-17f9-4438-b179-2d716e4cc295") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:37.892028 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:37.891998    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:37.892173 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.892156    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:37.892270 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.892174    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:37.892270 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.892187    2572 projected.go:196] Error preparing data for projected volume kube-api-access-tm58n for pod openshift-network-diagnostics/network-check-target-ddb52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:37.892270 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:37.892241    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n podName:cc75c925-0a50-440b-8def-b991e61e560e nodeName:}" failed. No retries permitted until 2026-07-16 21:15:38.89222775 +0000 UTC m=+4.403387475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm58n" (UniqueName: "kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n") pod "network-check-target-ddb52" (UID: "cc75c925-0a50-440b-8def-b991e61e560e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:38.193847 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:38.193752    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:10:37 +0000 UTC" deadline="2028-04-15 07:06:13.38408826 +0000 UTC"
Jul 16 21:15:38.193847 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:38.193854    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15321h50m35.190248019s"
Jul 16 21:15:38.244217 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:38.244180    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:38.244217 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:38.244202    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:38.244414 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.244304    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:38.244516 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.244489    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:38.360451 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:38.360404    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc865333f_a4ab_4780_80ce_7463b73a6acc.slice/crio-3b77628dbd4d0bf20789ec40ad7c0f9fb4b854b9abf1f6e1aa37d501fc02baf7 WatchSource:0}: Error finding container 3b77628dbd4d0bf20789ec40ad7c0f9fb4b854b9abf1f6e1aa37d501fc02baf7: Status 404 returned error can't find the container with id 3b77628dbd4d0bf20789ec40ad7c0f9fb4b854b9abf1f6e1aa37d501fc02baf7
Jul 16 21:15:38.362735 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:38.362687    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b58d65_9c6a_43c1_b913_981279a5ace0.slice/crio-50d2f5090f9e8d8799f62df6607493f1b3e1280ea7d0bcf8f8930e78eb0b1b6c WatchSource:0}: Error finding container 50d2f5090f9e8d8799f62df6607493f1b3e1280ea7d0bcf8f8930e78eb0b1b6c: Status 404 returned error can't find the container with id 50d2f5090f9e8d8799f62df6607493f1b3e1280ea7d0bcf8f8930e78eb0b1b6c
Jul 16 21:15:38.365455 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:38.365368    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4715f5_c6d4_46e5_9918_9e3f3f1cb179.slice/crio-a77f3eb443ccdddb3c152408bb10acef11c6f612de7e6d322eab993db41937de WatchSource:0}: Error finding container a77f3eb443ccdddb3c152408bb10acef11c6f612de7e6d322eab993db41937de: Status 404 returned error can't find the container with id a77f3eb443ccdddb3c152408bb10acef11c6f612de7e6d322eab993db41937de
Jul 16 21:15:38.366070 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:38.365959    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9229aed3_7c79_40f2_8886_21024b1b8ec9.slice/crio-f591899fd518230fe133384108882cfce3f8ba57c81a1e5a3fd4d02d0737fb72 WatchSource:0}: Error finding container f591899fd518230fe133384108882cfce3f8ba57c81a1e5a3fd4d02d0737fb72: Status 404 returned error can't find the container with id f591899fd518230fe133384108882cfce3f8ba57c81a1e5a3fd4d02d0737fb72
Jul 16 21:15:38.367129 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:38.367101    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8572761_a70c_4a5c_a04a_2ffc3ccec888.slice/crio-89f08321bb2dd5ecf340527601bcc331640a8fd51baecf13db8d9583e3d86237 WatchSource:0}: Error finding container 89f08321bb2dd5ecf340527601bcc331640a8fd51baecf13db8d9583e3d86237: Status 404 returned error can't find the container with id 89f08321bb2dd5ecf340527601bcc331640a8fd51baecf13db8d9583e3d86237
Jul 16 21:15:38.369254 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:38.369173    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69d0fe7a_f6b7_43c4_a258_31bf4790975e.slice/crio-412fff967b434b3688d571a4bc60e87e8cb4a917ad937a93f91647d57c8a1313 WatchSource:0}: Error finding container 412fff967b434b3688d571a4bc60e87e8cb4a917ad937a93f91647d57c8a1313: Status 404 returned error can't find the container with id 412fff967b434b3688d571a4bc60e87e8cb4a917ad937a93f91647d57c8a1313
Jul 16 21:15:38.372489 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:15:38.372465    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e6e6771_9ce2_4db7_9d34_e9bfba359c4a.slice/crio-df4fda9364c2375136c797476d1b3f53356deca84aa2a5fe7ddd71a5a160e10e WatchSource:0}: Error finding container df4fda9364c2375136c797476d1b3f53356deca84aa2a5fe7ddd71a5a160e10e: Status 404 returned error can't find the container with id df4fda9364c2375136c797476d1b3f53356deca84aa2a5fe7ddd71a5a160e10e
Jul 16 21:15:38.798859 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:38.798517    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:38.798859 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:38.798787    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:38.798859 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.798696    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:38.799135 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.798876    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret podName:61292e97-d6af-40aa-9f6b-c944ebe621b3 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:40.798856356 +0000 UTC m=+6.310016085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret") pod "global-pull-secret-syncer-zjlgl" (UID: "61292e97-d6af-40aa-9f6b-c944ebe621b3") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:38.799135 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.798906    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:38.799135 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.798948    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs podName:885f0bbb-17f9-4438-b179-2d716e4cc295 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:40.798935927 +0000 UTC m=+6.310095654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs") pod "network-metrics-daemon-2wwbb" (UID: "885f0bbb-17f9-4438-b179-2d716e4cc295") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:38.899380 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:38.899327    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:38.899591 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.899502    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:38.899591 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.899521    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:38.899591 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.899534    2572 projected.go:196] Error preparing data for projected volume kube-api-access-tm58n for pod openshift-network-diagnostics/network-check-target-ddb52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:38.899591 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:38.899601    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n podName:cc75c925-0a50-440b-8def-b991e61e560e nodeName:}" failed. No retries permitted until 2026-07-16 21:15:40.899573939 +0000 UTC m=+6.410733667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm58n" (UniqueName: "kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n") pod "network-check-target-ddb52" (UID: "cc75c925-0a50-440b-8def-b991e61e560e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:39.194493 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.194447    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 21:10:37 +0000 UTC" deadline="2028-01-21 23:34:02.468083841 +0000 UTC"
Jul 16 21:15:39.194493 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.194492    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13298h18m23.273598342s"
Jul 16 21:15:39.246864 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.246333    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:39.246864 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:39.246496    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:39.269198 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.269159    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerStarted","Data":"412fff967b434b3688d571a4bc60e87e8cb4a917ad937a93f91647d57c8a1313"}
Jul 16 21:15:39.275976 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.275937    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pnzkq" event={"ID":"9229aed3-7c79-40f2-8886-21024b1b8ec9","Type":"ContainerStarted","Data":"f591899fd518230fe133384108882cfce3f8ba57c81a1e5a3fd4d02d0737fb72"}
Jul 16 21:15:39.302704 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.302665    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"5d861383d4ed39d3b867758912cccf400c05c3f88559b2f21741be6af5f3278f"}
Jul 16 21:15:39.318891 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.318857    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qc6g5" event={"ID":"d8572761-a70c-4a5c-a04a-2ffc3ccec888","Type":"ContainerStarted","Data":"89f08321bb2dd5ecf340527601bcc331640a8fd51baecf13db8d9583e3d86237"}
Jul 16 21:15:39.327289 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.327251    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9fw8z" event={"ID":"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179","Type":"ContainerStarted","Data":"a77f3eb443ccdddb3c152408bb10acef11c6f612de7e6d322eab993db41937de"}
Jul 16 21:15:39.329888 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.329851    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mnn2h" event={"ID":"d7b58d65-9c6a-43c1-b913-981279a5ace0","Type":"ContainerStarted","Data":"50d2f5090f9e8d8799f62df6607493f1b3e1280ea7d0bcf8f8930e78eb0b1b6c"}
Jul 16 21:15:39.341799 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.341759    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp" event={"ID":"c865333f-a4ab-4780-80ce-7463b73a6acc","Type":"ContainerStarted","Data":"3b77628dbd4d0bf20789ec40ad7c0f9fb4b854b9abf1f6e1aa37d501fc02baf7"}
Jul 16 21:15:39.352009 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.351921    2572 generic.go:356] "Generic (PLEG): container finished" podID="a050b69ff95c7f4d535f7563e4b39735" containerID="a1bb362170e3b6b69c02d7fd9441f29b8f855447fdd333f1c77e6679b6f36103" exitCode=0
Jul 16 21:15:39.352165 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.352024    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal" event={"ID":"a050b69ff95c7f4d535f7563e4b39735","Type":"ContainerDied","Data":"a1bb362170e3b6b69c02d7fd9441f29b8f855447fdd333f1c77e6679b6f36103"}
Jul 16 21:15:39.390443 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.390389    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal" event={"ID":"5112854b758de5dd94996dccba2987eb","Type":"ContainerStarted","Data":"bf95d0196b6f5f47e29455bf4d4aa1e52562650e8a1bd45daa19e015bed0502b"}
Jul 16 21:15:39.407036 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.406924    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zwmb7" event={"ID":"a5c76402-3101-42e3-9af0-72d87c627513","Type":"ContainerStarted","Data":"6e33b236e64ff4ef55b55c14c47679c26effb2250b7e3dc327ad22bf73c66ac2"}
Jul 16 21:15:39.417563 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.417062    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-67.ec2.internal" podStartSLOduration=3.417040774 podStartE2EDuration="3.417040774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:39.415265926 +0000 UTC m=+4.926425658" watchObservedRunningTime="2026-07-16 21:15:39.417040774 +0000 UTC m=+4.928200611"
Jul 16 21:15:39.417563 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:39.417490    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zchn6" event={"ID":"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a","Type":"ContainerStarted","Data":"df4fda9364c2375136c797476d1b3f53356deca84aa2a5fe7ddd71a5a160e10e"}
Jul 16 21:15:40.244370 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:40.243640    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:40.244370 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.243783    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:40.244370 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:40.244201    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:40.244370 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.244296    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:40.440011 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:40.439896    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal" event={"ID":"a050b69ff95c7f4d535f7563e4b39735","Type":"ContainerStarted","Data":"9a3d7149404ab4838d93313e121c2ac96a5fe4116a68a7982ca1d01a5f6b6d03"}
Jul 16 21:15:40.815329 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:40.815258    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:40.815537 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:40.815374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:40.815604 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.815544    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:40.815659 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.815607    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs podName:885f0bbb-17f9-4438-b179-2d716e4cc295 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:44.815588496 +0000 UTC m=+10.326748227 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs") pod "network-metrics-daemon-2wwbb" (UID: "885f0bbb-17f9-4438-b179-2d716e4cc295") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:40.815721 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.815680    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:40.815721 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.815711    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret podName:61292e97-d6af-40aa-9f6b-c944ebe621b3 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:44.815701259 +0000 UTC m=+10.326860982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret") pod "global-pull-secret-syncer-zjlgl" (UID: "61292e97-d6af-40aa-9f6b-c944ebe621b3") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:40.916987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:40.916326    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:40.916987 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.916551    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:40.916987 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.916572    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:40.916987 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.916585    2572 projected.go:196] Error preparing data for projected volume kube-api-access-tm58n for pod openshift-network-diagnostics/network-check-target-ddb52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:40.916987 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:40.916644    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n podName:cc75c925-0a50-440b-8def-b991e61e560e nodeName:}" failed. No retries permitted until 2026-07-16 21:15:44.916624083 +0000 UTC m=+10.427783797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm58n" (UniqueName: "kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n") pod "network-check-target-ddb52" (UID: "cc75c925-0a50-440b-8def-b991e61e560e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:41.245733 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:41.245640    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:41.246162 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:41.245777    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:42.244088 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:42.244054    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:42.244281 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:42.244190    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:42.244281 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:42.244259    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:42.244436 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:42.244366    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:43.244044 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:43.243963    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:43.244503 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:43.244107    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:44.244132 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:44.244054    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:44.244132 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:44.244082    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:44.244670 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.244232    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:44.244670 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.244415    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:44.853272 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:44.853126    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:44.853272 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:44.853181    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:44.853566 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.853365    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:44.853566 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.853432    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs podName:885f0bbb-17f9-4438-b179-2d716e4cc295 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:52.853413255 +0000 UTC m=+18.364572985 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs") pod "network-metrics-daemon-2wwbb" (UID: "885f0bbb-17f9-4438-b179-2d716e4cc295") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:44.853928 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.853825    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:44.853928 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.853877    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret podName:61292e97-d6af-40aa-9f6b-c944ebe621b3 nodeName:}" failed. No retries permitted until 2026-07-16 21:15:52.853860979 +0000 UTC m=+18.365020697 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret") pod "global-pull-secret-syncer-zjlgl" (UID: "61292e97-d6af-40aa-9f6b-c944ebe621b3") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:44.954525 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:44.954399    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:44.954717 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.954596    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:44.954717 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.954624    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:44.954717 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.954637    2572 projected.go:196] Error preparing data for projected volume kube-api-access-tm58n for pod openshift-network-diagnostics/network-check-target-ddb52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:44.954717 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:44.954698    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n podName:cc75c925-0a50-440b-8def-b991e61e560e nodeName:}" failed. No retries permitted until 2026-07-16 21:15:52.954679438 +0000 UTC m=+18.465839163 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm58n" (UniqueName: "kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n") pod "network-check-target-ddb52" (UID: "cc75c925-0a50-440b-8def-b991e61e560e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:45.245269 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:45.245170    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:45.245802 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:45.245296    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:46.243580 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:46.243540    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:46.243741 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:46.243541    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:46.243741 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:46.243690    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:46.243741 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:46.243736    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:47.244558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:47.244393    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:47.244558 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:47.244528    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:48.243461 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:48.243425    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:48.243653 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:48.243548    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:48.243653 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:48.243607    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:48.243806 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:48.243783    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:49.243547 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:49.243510    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:49.244043 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:49.243656    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:50.243526 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:50.243485    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:50.243715 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:50.243550    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:50.243715 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:50.243677    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:50.244111 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:50.243804    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:51.244346 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:51.244301    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:51.244860 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:51.244451    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:52.244420 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:52.244375    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:52.244880 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:52.244385    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:52.244880 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:52.244536    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:52.244880 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:52.244596    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:52.920915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:52.920823    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:52.920915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:52.920866    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:52.921121 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:52.921011    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:52.921121 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:52.921043    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:52.921121 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:52.921087    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret podName:61292e97-d6af-40aa-9f6b-c944ebe621b3 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:08.921066329 +0000 UTC m=+34.432226040 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret") pod "global-pull-secret-syncer-zjlgl" (UID: "61292e97-d6af-40aa-9f6b-c944ebe621b3") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:15:52.921121 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:52.921110    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs podName:885f0bbb-17f9-4438-b179-2d716e4cc295 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:08.921099861 +0000 UTC m=+34.432259577 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs") pod "network-metrics-daemon-2wwbb" (UID: "885f0bbb-17f9-4438-b179-2d716e4cc295") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:15:53.021805 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:53.021763    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:53.022015 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:53.021984    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:15:53.022015 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:53.022014    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:15:53.022183 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:53.022028    2572 projected.go:196] Error preparing data for projected volume kube-api-access-tm58n for pod openshift-network-diagnostics/network-check-target-ddb52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:53.022183 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:53.022096    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n podName:cc75c925-0a50-440b-8def-b991e61e560e nodeName:}" failed. No retries permitted until 2026-07-16 21:16:09.022077598 +0000 UTC m=+34.533237310 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm58n" (UniqueName: "kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n") pod "network-check-target-ddb52" (UID: "cc75c925-0a50-440b-8def-b991e61e560e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:15:53.244406 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:53.244310    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:53.244572 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:53.244458    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:54.243694 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:54.243659    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:54.243864 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:54.243713    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:54.243864 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:54.243820    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:54.243978 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:54.243958    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:55.244757 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:55.244717    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:55.245203 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:55.244840    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:56.243370 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:56.243325    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:56.243559 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:56.243323    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:56.243559 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:56.243480    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:56.243655 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:56.243562    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:57.243748 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.243322    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:57.244403 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:57.243864    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:57.475063 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.474979    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pnzkq" event={"ID":"9229aed3-7c79-40f2-8886-21024b1b8ec9","Type":"ContainerStarted","Data":"dd83421f5f4a7a648c9aee760f8927121d93e280356043e2819c5c39f42f2356"}
Jul 16 21:15:57.480538 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.480504    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"72ed9181f3b78d75914934f4066c1cf398f6cbf0cf30c4431296d8b0c8520b18"}
Jul 16 21:15:57.480695 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.480594    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"80356baff96dabcf48576427ac863488bf7b18564d650e9e82322586fe5840ce"}
Jul 16 21:15:57.480695 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.480639    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"ffe1819d8cb873dfa73914fcf16c8d952e3c7a1b3515984b3bc46338f8f96e39"}
Jul 16 21:15:57.480695 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.480681    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"8d6ec14d8bae3ae7c478006a58ab19f0cd9b2d6f028259836ff9bac1ef35bef5"}
Jul 16 21:15:57.480862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.480730    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"0fd9646d374978a955bcdf8f56b11c009ca4003954ba8538de3ee65c5c1caba0"}
Jul 16 21:15:57.480862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.480778    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"e7417302ef1dd8cee02cb6f009d2fd279414d2d873fe36f2a3af4cc2184ec2e3"}
Jul 16 21:15:57.482178 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.482152    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qc6g5" event={"ID":"d8572761-a70c-4a5c-a04a-2ffc3ccec888","Type":"ContainerStarted","Data":"c75fc33b5fa5d33e18a32d88400057b8e846bce2b188ca202d35900f46e9b0b2"}
Jul 16 21:15:57.483756 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.483720    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9fw8z" event={"ID":"cb4715f5-c6d4-46e5-9918-9e3f3f1cb179","Type":"ContainerStarted","Data":"0d7db5624c0c0969b06749d4b90a75e26c55e6467520dccc2d2a4ebfce32be1e"}
Jul 16 21:15:57.485181 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.485147    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mnn2h" event={"ID":"d7b58d65-9c6a-43c1-b913-981279a5ace0","Type":"ContainerStarted","Data":"18b2294a2512ef92b7abd7d6209f97aad159b0b1162858b896223045851c8398"}
Jul 16 21:15:57.486588 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.486564    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp" event={"ID":"c865333f-a4ab-4780-80ce-7463b73a6acc","Type":"ContainerStarted","Data":"9e66325b0444d7fd3aa12a84bf118361121e6b973a3f0c201db503b1a9b36260"}
Jul 16 21:15:57.488079 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.488056    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zchn6" event={"ID":"6e6e6771-9ce2-4db7-9d34-e9bfba359c4a","Type":"ContainerStarted","Data":"c8472bc85b0de740cda1998ac1af5dbf2c25740346474e5049001a33b0eafb39"}
Jul 16 21:15:57.489505 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.489484    2572 generic.go:356] "Generic (PLEG): container finished" podID="69d0fe7a-f6b7-43c4-a258-31bf4790975e" containerID="a0362f7a9f51d5df8c5c2bb524a3040f899e285296df929f82986a685cccb2f8" exitCode=0
Jul 16 21:15:57.489587 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.489511    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerDied","Data":"a0362f7a9f51d5df8c5c2bb524a3040f899e285296df929f82986a685cccb2f8"}
Jul 16 21:15:57.500281 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.500219    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pnzkq" podStartSLOduration=4.187512218 podStartE2EDuration="22.500200621s" totalImagesPullingTime="18.312688403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:57.499465874 +0000 UTC m=+23.010625622" watchObservedRunningTime="2026-07-16 21:15:57.500200621 +0000 UTC m=+23.011360355"
Jul 16 21:15:57.500431 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.500378    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-67.ec2.internal" podStartSLOduration=21.500367443000002 podStartE2EDuration="21.500367443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:40.461764615 +0000 UTC m=+5.972924350" watchObservedRunningTime="2026-07-16 21:15:57.500367443 +0000 UTC m=+23.011527172"
Jul 16 21:15:57.522031 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.521981    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9fw8z" podStartSLOduration=4.152601284 podStartE2EDuration="22.521966482s" totalImagesPullingTime="18.369365198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:57.521594608 +0000 UTC m=+23.032754343" watchObservedRunningTime="2026-07-16 21:15:57.521966482 +0000 UTC m=+23.033126215"
Jul 16 21:15:57.557670 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.557618    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qc6g5" podStartSLOduration=8.604073108 podStartE2EDuration="22.557597713s" totalImagesPullingTime="13.953524605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:57.540605589 +0000 UTC m=+23.051765323" watchObservedRunningTime="2026-07-16 21:15:57.557597713 +0000 UTC m=+23.068757448"
Jul 16 21:15:57.582928 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.582864    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zchn6" podStartSLOduration=4.2778300510000005 podStartE2EDuration="22.582846259s" totalImagesPullingTime="18.305016208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:57.55782417 +0000 UTC m=+23.068983904" watchObservedRunningTime="2026-07-16 21:15:57.582846259 +0000 UTC m=+23.094005983"
Jul 16 21:15:57.602255 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:57.602209    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mnn2h" podStartSLOduration=8.644101271 podStartE2EDuration="22.602196146s" totalImagesPullingTime="13.958094875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:57.60210474 +0000 UTC m=+23.113264474" watchObservedRunningTime="2026-07-16 21:15:57.602196146 +0000 UTC m=+23.113355879"
Jul 16 21:15:58.231643 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:58.231611    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 21:15:58.243444 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:58.243407    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:15:58.243576 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:58.243428    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:15:58.243576 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:58.243553    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:15:58.243688 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:58.243634    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:15:58.493949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:58.493864    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp" event={"ID":"c865333f-a4ab-4780-80ce-7463b73a6acc","Type":"ContainerStarted","Data":"21acab6b11c1fcb3fa98f324b5cc03eaec1fdda941cd92c2c4d79b6bc709d7c4"}
Jul 16 21:15:58.495645 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:58.495613    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zwmb7" event={"ID":"a5c76402-3101-42e3-9af0-72d87c627513","Type":"ContainerStarted","Data":"816d55b95f96b291fcb0ec75ba92316fb052d64bfd80895f6844d55c18493edf"}
Jul 16 21:15:59.136931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:59.136820    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T21:15:58.231636409Z","UUID":"85ddee8a-8712-4614-9e80-4ee5e25b338e","Handler":null,"Name":"","Endpoint":""}
Jul 16 21:15:59.139149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:59.139115    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 21:15:59.139315 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:59.139168    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 21:15:59.244183 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:59.244148    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:15:59.244360 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:15:59.244287    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:15:59.462681 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:59.462649    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:59.463230 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:59.463212    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:15:59.482552 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:59.482511    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zwmb7" podStartSLOduration=6.175962551 podStartE2EDuration="24.482496671s" totalImagesPullingTime="18.30653412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:58.536909603 +0000 UTC m=+24.048069338" watchObservedRunningTime="2026-07-16 21:15:59.482496671 +0000 UTC m=+24.993656405"
Jul 16 21:15:59.500776 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:15:59.500742    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp" event={"ID":"c865333f-a4ab-4780-80ce-7463b73a6acc","Type":"ContainerStarted","Data":"8727be02185eb25fd6bb04af58be9470ab99ea1f5c1f85d41ebd3bceea5ab2ea"}
Jul 16 21:16:00.243879 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:00.243695    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:00.244142 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:00.243699    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:00.244142 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:00.243994    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:16:00.244142 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:00.244077    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:16:00.506632 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:00.506542    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"cbea62c3225ae2027ea49c7ebb01d082e361c2544166661444330f5864d834fe"}
Jul 16 21:16:00.506632 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:00.506619    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 16 21:16:00.538962 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:00.538899    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zd9zp" podStartSLOduration=4.827847133 podStartE2EDuration="25.538878089s" totalImagesPullingTime="20.711030956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:00.538717019 +0000 UTC m=+26.049876753" watchObservedRunningTime="2026-07-16 21:16:00.538878089 +0000 UTC m=+26.050037824"
Jul 16 21:16:01.244428 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:01.244389    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:01.244642 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:01.244511    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:16:01.962598 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:01.962510    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:16:01.963155 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:01.962668    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 16 21:16:01.963228 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:01.963167    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mnn2h"
Jul 16 21:16:02.243548 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:02.243517    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:02.243696 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:02.243554    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:02.243696 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:02.243644    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:16:02.243839 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:02.243817    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:16:02.513753 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:02.513467    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerStarted","Data":"d63c0cd90a598be90b281d5d979ec496c15093aaf127c5de0916cbb27eb11349"}
Jul 16 21:16:02.517270 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:02.517231    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" event={"ID":"74cb201b-9a56-4551-8a30-dd8dfc1f2579","Type":"ContainerStarted","Data":"f6e6411096024028d46e778651c9a348085b96ea00157d10417e97601befae68"}
Jul 16 21:16:03.243329 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.243260    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:03.243931 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:03.243372    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:16:03.521135 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.521059    2572 generic.go:356] "Generic (PLEG): container finished" podID="69d0fe7a-f6b7-43c4-a258-31bf4790975e" containerID="d63c0cd90a598be90b281d5d979ec496c15093aaf127c5de0916cbb27eb11349" exitCode=0
Jul 16 21:16:03.521365 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.521144    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerDied","Data":"d63c0cd90a598be90b281d5d979ec496c15093aaf127c5de0916cbb27eb11349"}
Jul 16 21:16:03.521683 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.521648    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:16:03.521788 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.521726    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:16:03.521788 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.521774    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:16:03.537287 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.537260    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:16:03.537449 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.537387    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:16:03.567337 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:03.567286    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz" podStartSLOduration=10.182890256 podStartE2EDuration="28.567268911s" totalImagesPullingTime="18.384378655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:02.595269809 +0000 UTC m=+28.106429543" watchObservedRunningTime="2026-07-16 21:16:03.567268911 +0000 UTC m=+29.078428645"
Jul 16 21:16:04.244318 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:04.244274    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:04.244758 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:04.244273    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:04.244758 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:04.244421    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:16:04.244758 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:04.244472    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:16:05.244370 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:05.244331    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:05.244783 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:05.244438    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:16:05.527627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:05.527533    2572 generic.go:356] "Generic (PLEG): container finished" podID="69d0fe7a-f6b7-43c4-a258-31bf4790975e" containerID="456cac29285f2ed25c89cb7faf73f6904697f3de93f964182166ecce5014cf6c" exitCode=0
Jul 16 21:16:05.527627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:05.527615    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerDied","Data":"456cac29285f2ed25c89cb7faf73f6904697f3de93f964182166ecce5014cf6c"}
Jul 16 21:16:06.243389 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:06.243329    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:06.243543 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:06.243345    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:06.243543 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:06.243493    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:16:06.243543 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:06.243518    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:16:07.243657 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:07.243618    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:07.244131 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:07.243729    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:16:07.534506 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:07.534422    2572 generic.go:356] "Generic (PLEG): container finished" podID="69d0fe7a-f6b7-43c4-a258-31bf4790975e" containerID="b89bde72fe962a24ca5476984d81689fc60f8b78acf625eae028273176c31d9c" exitCode=0
Jul 16 21:16:07.534506 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:07.534471    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerDied","Data":"b89bde72fe962a24ca5476984d81689fc60f8b78acf625eae028273176c31d9c"}
Jul 16 21:16:08.243963 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:08.243930    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:08.244473 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:08.243941    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:08.244473 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:08.244061    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:16:08.244473 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:08.244151    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:16:08.935369 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:08.935328    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:08.935547 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:08.935394    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:08.935547 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:08.935447    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 21:16:08.935547 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:08.935513    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:16:08.935547 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:08.935522    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret podName:61292e97-d6af-40aa-9f6b-c944ebe621b3 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:40.935500872 +0000 UTC m=+66.446660586 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret") pod "global-pull-secret-syncer-zjlgl" (UID: "61292e97-d6af-40aa-9f6b-c944ebe621b3") : object "kube-system"/"original-pull-secret" not registered
Jul 16 21:16:08.935759 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:08.935560    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs podName:885f0bbb-17f9-4438-b179-2d716e4cc295 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:40.935544384 +0000 UTC m=+66.446704101 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs") pod "network-metrics-daemon-2wwbb" (UID: "885f0bbb-17f9-4438-b179-2d716e4cc295") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 21:16:09.036113 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:09.036067    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:09.036319 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:09.036223    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 21:16:09.036319 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:09.036253    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 21:16:09.036319 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:09.036264    2572 projected.go:196] Error preparing data for projected volume kube-api-access-tm58n for pod openshift-network-diagnostics/network-check-target-ddb52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:16:09.036499 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:09.036330    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n podName:cc75c925-0a50-440b-8def-b991e61e560e nodeName:}" failed. No retries permitted until 2026-07-16 21:16:41.036302445 +0000 UTC m=+66.547462177 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm58n" (UniqueName: "kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n") pod "network-check-target-ddb52" (UID: "cc75c925-0a50-440b-8def-b991e61e560e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 21:16:09.244081 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:09.244002    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:09.244535 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:09.244141    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:16:10.089206 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:10.088971    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ddb52"]
Jul 16 21:16:10.089398 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:10.089344    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:10.089491 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:10.089469    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:16:10.089721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:10.089690    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2wwbb"]
Jul 16 21:16:10.089813 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:10.089805    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:10.089930 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:10.089907    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:16:10.107852 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:10.107814    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zjlgl"]
Jul 16 21:16:10.108029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:10.107944    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:10.108082 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:10.108037    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:16:11.243319 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:11.243283    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:11.243874 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:11.243283    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:11.243874 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:11.243455    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:16:11.243874 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:11.243496    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:16:12.243513 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:12.243477    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:12.244039 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:12.243592    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:16:12.558857 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:12.558763    2572 generic.go:356] "Generic (PLEG): container finished" podID="69d0fe7a-f6b7-43c4-a258-31bf4790975e" containerID="cef18399b4c662f69106efea233974cff7d30f660ae2e11aa06248e7245e41ce" exitCode=0
Jul 16 21:16:12.558857 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:12.558806    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerDied","Data":"cef18399b4c662f69106efea233974cff7d30f660ae2e11aa06248e7245e41ce"}
Jul 16 21:16:13.246620 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:13.246591    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:13.246620 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:13.246613    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:13.247111 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:13.246691    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ddb52" podUID="cc75c925-0a50-440b-8def-b991e61e560e"
Jul 16 21:16:13.247111 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:13.246810    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2wwbb" podUID="885f0bbb-17f9-4438-b179-2d716e4cc295"
Jul 16 21:16:13.564543 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:13.564433    2572 generic.go:356] "Generic (PLEG): container finished" podID="69d0fe7a-f6b7-43c4-a258-31bf4790975e" containerID="d615fc576262ab2051eca8f44062defd2eb02e7f69b2af58ba6497d3b9b565e1" exitCode=0
Jul 16 21:16:13.564543 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:13.564505    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerDied","Data":"d615fc576262ab2051eca8f44062defd2eb02e7f69b2af58ba6497d3b9b565e1"}
Jul 16 21:16:14.244130 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.244094    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:14.244274 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:14.244211    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zjlgl" podUID="61292e97-d6af-40aa-9f6b-c944ebe621b3"
Jul 16 21:16:14.320954 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.320919    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-67.ec2.internal" event="NodeReady"
Jul 16 21:16:14.321332 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.321088    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 21:16:14.444898 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.444818    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2r98p"]
Jul 16 21:16:14.469960 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.469922    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pc7xc"]
Jul 16 21:16:14.470120 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.470040    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.479494 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.479469    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 21:16:14.479640 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.479469    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 21:16:14.479640 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.479506    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 21:16:14.479867 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.479853    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 21:16:14.480323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.480298    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m9h62\""
Jul 16 21:16:14.494155 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.494112    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pc7xc"]
Jul 16 21:16:14.494155 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.494161    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2r98p"]
Jul 16 21:16:14.494399 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.494260    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.500293 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.500267    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pkgpg"]
Jul 16 21:16:14.511946 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.511921    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 21:16:14.512862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.512844    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 21:16:14.524818 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.524788    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dhx8d\""
Jul 16 21:16:14.525927 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.525909    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pkgpg"
Jul 16 21:16:14.533747 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.533724    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-blkm2\""
Jul 16 21:16:14.533863 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.533762    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 21:16:14.533863 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.533762    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 21:16:14.533863 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.533762    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 21:16:14.544529 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.544502    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pkgpg"]
Jul 16 21:16:14.570109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.570070    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kn78t" event={"ID":"69d0fe7a-f6b7-43c4-a258-31bf4790975e","Type":"ContainerStarted","Data":"4489fdca5a4e9dba695c23f573dcc3e3569744a52a4335388b10dff4b19de18b"}
Jul 16 21:16:14.576094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576065    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kldt\" (UniqueName: \"kubernetes.io/projected/db58efe2-e346-46d0-9827-86078b2c445a-kube-api-access-7kldt\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.576234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576103    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72a05823-0e7c-4d2b-b6be-2a6972a2827d-crio-socket\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.576234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576123    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72a05823-0e7c-4d2b-b6be-2a6972a2827d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.576234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576141    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9jt7\" (UniqueName: \"kubernetes.io/projected/72a05823-0e7c-4d2b-b6be-2a6972a2827d-kube-api-access-g9jt7\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.576234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576161    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db58efe2-e346-46d0-9827-86078b2c445a-config-volume\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.576234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576193    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db58efe2-e346-46d0-9827-86078b2c445a-tmp-dir\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.576515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576275    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72a05823-0e7c-4d2b-b6be-2a6972a2827d-data-volume\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.576515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576339    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72a05823-0e7c-4d2b-b6be-2a6972a2827d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.576515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.576399    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db58efe2-e346-46d0-9827-86078b2c445a-metrics-tls\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.677066 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677033    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72a05823-0e7c-4d2b-b6be-2a6972a2827d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.677066 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677070    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db58efe2-e346-46d0-9827-86078b2c445a-metrics-tls\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.677271 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677245    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/819d0098-4e1a-4594-be74-a9307021a5ce-cert\") pod \"ingress-canary-pkgpg\" (UID: \"819d0098-4e1a-4594-be74-a9307021a5ce\") " pod="openshift-ingress-canary/ingress-canary-pkgpg"
Jul 16 21:16:14.677384 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677334    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kldt\" (UniqueName: \"kubernetes.io/projected/db58efe2-e346-46d0-9827-86078b2c445a-kube-api-access-7kldt\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.677508 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677401    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72a05823-0e7c-4d2b-b6be-2a6972a2827d-crio-socket\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.677508 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677428    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72a05823-0e7c-4d2b-b6be-2a6972a2827d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.677508 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677473    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9jt7\" (UniqueName: \"kubernetes.io/projected/72a05823-0e7c-4d2b-b6be-2a6972a2827d-kube-api-access-g9jt7\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.677508 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677498    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db58efe2-e346-46d0-9827-86078b2c445a-config-volume\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.677712 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677522    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db58efe2-e346-46d0-9827-86078b2c445a-tmp-dir\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.677712 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677606    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4r7p\" (UniqueName: \"kubernetes.io/projected/819d0098-4e1a-4594-be74-a9307021a5ce-kube-api-access-b4r7p\") pod \"ingress-canary-pkgpg\" (UID: \"819d0098-4e1a-4594-be74-a9307021a5ce\") " pod="openshift-ingress-canary/ingress-canary-pkgpg"
Jul 16 21:16:14.677712 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.677638    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72a05823-0e7c-4d2b-b6be-2a6972a2827d-data-volume\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.678100 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.678080    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/72a05823-0e7c-4d2b-b6be-2a6972a2827d-crio-socket\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.678188 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.678099    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db58efe2-e346-46d0-9827-86078b2c445a-tmp-dir\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.678188 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.678112    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/72a05823-0e7c-4d2b-b6be-2a6972a2827d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.678282 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.678224    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/72a05823-0e7c-4d2b-b6be-2a6972a2827d-data-volume\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.678423 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.678385    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kn78t" podStartSLOduration=10.322974136 podStartE2EDuration="39.678373776s" totalImagesPullingTime="29.35539964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:14.675228239 +0000 UTC m=+40.186387997" watchObservedRunningTime="2026-07-16 21:16:14.678373776 +0000 UTC m=+40.189533500"
Jul 16 21:16:14.681085 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.681064    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db58efe2-e346-46d0-9827-86078b2c445a-metrics-tls\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.681179 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.681096    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/72a05823-0e7c-4d2b-b6be-2a6972a2827d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.691721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.691693    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db58efe2-e346-46d0-9827-86078b2c445a-config-volume\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.705179 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.705108    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kldt\" (UniqueName: \"kubernetes.io/projected/db58efe2-e346-46d0-9827-86078b2c445a-kube-api-access-7kldt\") pod \"dns-default-pc7xc\" (UID: \"db58efe2-e346-46d0-9827-86078b2c445a\") " pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.722117 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.722081    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9jt7\" (UniqueName: \"kubernetes.io/projected/72a05823-0e7c-4d2b-b6be-2a6972a2827d-kube-api-access-g9jt7\") pod \"insights-runtime-extractor-2r98p\" (UID: \"72a05823-0e7c-4d2b-b6be-2a6972a2827d\") " pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.778293 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.778252    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4r7p\" (UniqueName: \"kubernetes.io/projected/819d0098-4e1a-4594-be74-a9307021a5ce-kube-api-access-b4r7p\") pod \"ingress-canary-pkgpg\" (UID: \"819d0098-4e1a-4594-be74-a9307021a5ce\") " pod="openshift-ingress-canary/ingress-canary-pkgpg"
Jul 16 21:16:14.778456 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.778406    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/819d0098-4e1a-4594-be74-a9307021a5ce-cert\") pod \"ingress-canary-pkgpg\" (UID: \"819d0098-4e1a-4594-be74-a9307021a5ce\") " pod="openshift-ingress-canary/ingress-canary-pkgpg"
Jul 16 21:16:14.778456 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.778427    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2r98p"
Jul 16 21:16:14.780767 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.780749    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/819d0098-4e1a-4594-be74-a9307021a5ce-cert\") pod \"ingress-canary-pkgpg\" (UID: \"819d0098-4e1a-4594-be74-a9307021a5ce\") " pod="openshift-ingress-canary/ingress-canary-pkgpg"
Jul 16 21:16:14.803762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.803737    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:14.806564 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.806539    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4r7p\" (UniqueName: \"kubernetes.io/projected/819d0098-4e1a-4594-be74-a9307021a5ce-kube-api-access-b4r7p\") pod \"ingress-canary-pkgpg\" (UID: \"819d0098-4e1a-4594-be74-a9307021a5ce\") " pod="openshift-ingress-canary/ingress-canary-pkgpg"
Jul 16 21:16:14.834100 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:14.834066    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pkgpg"
Jul 16 21:16:15.000112 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.000078    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pc7xc"]
Jul 16 21:16:15.003991 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:15.003960    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb58efe2_e346_46d0_9827_86078b2c445a.slice/crio-a53815726c378839109c4ebc8926fc438e9aaafa212af54e5d05b73d59d58f94 WatchSource:0}: Error finding container a53815726c378839109c4ebc8926fc438e9aaafa212af54e5d05b73d59d58f94: Status 404 returned error can't find the container with id a53815726c378839109c4ebc8926fc438e9aaafa212af54e5d05b73d59d58f94
Jul 16 21:16:15.013384 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.013340    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2r98p"]
Jul 16 21:16:15.014876 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:15.014850    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72a05823_0e7c_4d2b_b6be_2a6972a2827d.slice/crio-4dcb4003fdb3f80046f393f19c0bc3b4abb4735f82946d88479515877dea81c7 WatchSource:0}: Error finding container 4dcb4003fdb3f80046f393f19c0bc3b4abb4735f82946d88479515877dea81c7: Status 404 returned error can't find the container with id 4dcb4003fdb3f80046f393f19c0bc3b4abb4735f82946d88479515877dea81c7
Jul 16 21:16:15.033252 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.033222    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pkgpg"]
Jul 16 21:16:15.036234 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:15.036208    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819d0098_4e1a_4594_be74_a9307021a5ce.slice/crio-189d009a6372cda35731ad5c61cc51ea9e354e352751ce6a33711e49e99dd715 WatchSource:0}: Error finding container 189d009a6372cda35731ad5c61cc51ea9e354e352751ce6a33711e49e99dd715: Status 404 returned error can't find the container with id 189d009a6372cda35731ad5c61cc51ea9e354e352751ce6a33711e49e99dd715
Jul 16 21:16:15.245809 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.245717    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:15.245957 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.245917    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:15.250347 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.250322    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 21:16:15.250502 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.250479    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 21:16:15.250502 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.250481    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzz4k\""
Jul 16 21:16:15.250583 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.250503    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 21:16:15.250630 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.250614    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tpjps\""
Jul 16 21:16:15.574462 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.574426    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2r98p" event={"ID":"72a05823-0e7c-4d2b-b6be-2a6972a2827d","Type":"ContainerStarted","Data":"4dcb4003fdb3f80046f393f19c0bc3b4abb4735f82946d88479515877dea81c7"}
Jul 16 21:16:15.575709 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.575683    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pkgpg" event={"ID":"819d0098-4e1a-4594-be74-a9307021a5ce","Type":"ContainerStarted","Data":"189d009a6372cda35731ad5c61cc51ea9e354e352751ce6a33711e49e99dd715"}
Jul 16 21:16:15.576814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:15.576788    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pc7xc" event={"ID":"db58efe2-e346-46d0-9827-86078b2c445a","Type":"ContainerStarted","Data":"a53815726c378839109c4ebc8926fc438e9aaafa212af54e5d05b73d59d58f94"}
Jul 16 21:16:16.243657 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.243617    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:16.250857 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.250635    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 21:16:16.505004 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.504914    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fdb985c9c-qm9ks"]
Jul 16 21:16:16.527968 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.527929    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.544567 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.544486    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fdb985c9c-qm9ks"]
Jul 16 21:16:16.546931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.546907    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 16 21:16:16.547220 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.547201    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 16 21:16:16.547341 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.547309    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 16 21:16:16.547573 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.547556    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 21:16:16.548392 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.548333    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 16 21:16:16.549393 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.549373    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 16 21:16:16.551126 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.551103    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 21:16:16.551428 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.551410    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ht7k4\""
Jul 16 21:16:16.554665 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.554644    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 16 21:16:16.695002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.694960    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-oauth-serving-cert\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.695002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.695026    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2btw2\" (UniqueName: \"kubernetes.io/projected/6fd5a03c-a188-4505-8f05-33bdd49300a6-kube-api-access-2btw2\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.695699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.695142    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-oauth-config\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.695699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.695181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-serving-cert\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.695699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.695224    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-config\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.695699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.695283    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-service-ca\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.695699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.695324    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-trusted-ca-bundle\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.796604 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.796570    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-config\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.796778 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.796645    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-service-ca\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.796778 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.796667    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-trusted-ca-bundle\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.796778 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.796698    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-oauth-serving-cert\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.796778 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.796729    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2btw2\" (UniqueName: \"kubernetes.io/projected/6fd5a03c-a188-4505-8f05-33bdd49300a6-kube-api-access-2btw2\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.796778 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.796774    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-oauth-config\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.797541 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.797160    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-serving-cert\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.797541 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.797484    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-config\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.798059 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.798034    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-trusted-ca-bundle\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.798167 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.798144    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-service-ca\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.798227 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.798176    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-oauth-serving-cert\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.799755 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.799712    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-oauth-config\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.800675 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.800603    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-serving-cert\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.811206 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.811166    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2btw2\" (UniqueName: \"kubernetes.io/projected/6fd5a03c-a188-4505-8f05-33bdd49300a6-kube-api-access-2btw2\") pod \"console-fdb985c9c-qm9ks\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") " pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:16.839936 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:16.839902    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:21.255894 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:21.255859    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fdb985c9c-qm9ks"]
Jul 16 21:16:22.125999 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:22.125969    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd5a03c_a188_4505_8f05_33bdd49300a6.slice/crio-f2bba753c5fda89bdd7738271a93b6f512b314858d1ff96b91d87139e6f4adcd WatchSource:0}: Error finding container f2bba753c5fda89bdd7738271a93b6f512b314858d1ff96b91d87139e6f4adcd: Status 404 returned error can't find the container with id f2bba753c5fda89bdd7738271a93b6f512b314858d1ff96b91d87139e6f4adcd
Jul 16 21:16:22.598429 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:22.598382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdb985c9c-qm9ks" event={"ID":"6fd5a03c-a188-4505-8f05-33bdd49300a6","Type":"ContainerStarted","Data":"f2bba753c5fda89bdd7738271a93b6f512b314858d1ff96b91d87139e6f4adcd"}
Jul 16 21:16:22.600624 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:22.600588    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pc7xc" event={"ID":"db58efe2-e346-46d0-9827-86078b2c445a","Type":"ContainerStarted","Data":"23ac7d77655fb71eb75b672acbc2f32f8e779846a5a677c68b16ab044ccd0cfb"}
Jul 16 21:16:22.600783 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:22.600656    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pc7xc" event={"ID":"db58efe2-e346-46d0-9827-86078b2c445a","Type":"ContainerStarted","Data":"fe77659797443db498e2b8f213e4579d88d7acb75a8bedfe03419799a158a72f"}
Jul 16 21:16:22.600783 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:22.600736    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:22.602239 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:22.602209    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pkgpg" event={"ID":"819d0098-4e1a-4594-be74-a9307021a5ce","Type":"ContainerStarted","Data":"56e57d1d9fcf38e7098114d8f8d1481d5bfdd538d4d3113dfc201278ed2d3cba"}
Jul 16 21:16:22.603921 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:22.603890    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2r98p" event={"ID":"72a05823-0e7c-4d2b-b6be-2a6972a2827d","Type":"ContainerStarted","Data":"908798c2e64779b7b528443fc032a4fee9ef4475d36f9a0922f217b9f0d8cf80"}
Jul 16 21:16:22.639707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:22.639652    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pc7xc" podStartSLOduration=1.5192161610000001 podStartE2EDuration="8.639636234s" totalImagesPullingTime="7.120420073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:22.639177444 +0000 UTC m=+48.150337179" watchObservedRunningTime="2026-07-16 21:16:22.639636234 +0000 UTC m=+48.150795957"
Jul 16 21:16:22.667692 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:22.667620    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pkgpg" podStartSLOduration=1.579769922 podStartE2EDuration="8.667596095s" totalImagesPullingTime="7.087826173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:22.66706894 +0000 UTC m=+48.178228675" watchObservedRunningTime="2026-07-16 21:16:22.667596095 +0000 UTC m=+48.178755827"
Jul 16 21:16:23.179527 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.179491    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"]
Jul 16 21:16:23.193273 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.193230    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.201759 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.201734    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 16 21:16:23.204875 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.204851    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 16 21:16:23.204987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.204923    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-p7zxg\""
Jul 16 21:16:23.205841 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.205810    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 21:16:23.208006 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.207985    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 21:16:23.216112 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.216086    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 21:16:23.216252 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.216192    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"]
Jul 16 21:16:23.347622 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.347583    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.347622 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.347631    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.347878 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.347663    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183b5c61-663b-423c-8b1b-e3c07131cb74-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.347878 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.347732    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnwbh\" (UniqueName: \"kubernetes.io/projected/183b5c61-663b-423c-8b1b-e3c07131cb74-kube-api-access-hnwbh\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.448797 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.448702    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.448797 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.448757    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.449018 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.448798    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183b5c61-663b-423c-8b1b-e3c07131cb74-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.449018 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.448827    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnwbh\" (UniqueName: \"kubernetes.io/projected/183b5c61-663b-423c-8b1b-e3c07131cb74-kube-api-access-hnwbh\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.449018 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:23.448879    2572 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 16 21:16:23.449018 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:23.448948    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-tls podName:183b5c61-663b-423c-8b1b-e3c07131cb74 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:23.948926893 +0000 UTC m=+49.460086605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-bx9qf" (UID: "183b5c61-663b-423c-8b1b-e3c07131cb74") : secret "prometheus-operator-tls" not found
Jul 16 21:16:23.449853 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.449827    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/183b5c61-663b-423c-8b1b-e3c07131cb74-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.451772 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.451747    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.489985 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.489920    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnwbh\" (UniqueName: \"kubernetes.io/projected/183b5c61-663b-423c-8b1b-e3c07131cb74-kube-api-access-hnwbh\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.953289 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.953249    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:23.957911 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:23.957849    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/183b5c61-663b-423c-8b1b-e3c07131cb74-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-bx9qf\" (UID: \"183b5c61-663b-423c-8b1b-e3c07131cb74\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:24.104699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:24.104656    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"
Jul 16 21:16:27.251605 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:27.251572    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf"]
Jul 16 21:16:27.823017 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:27.822980    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183b5c61_663b_423c_8b1b_e3c07131cb74.slice/crio-ffdd65a41c839c719946a8b3a3bbac686d7e7c96e4dda5223466d58e55dc8392 WatchSource:0}: Error finding container ffdd65a41c839c719946a8b3a3bbac686d7e7c96e4dda5223466d58e55dc8392: Status 404 returned error can't find the container with id ffdd65a41c839c719946a8b3a3bbac686d7e7c96e4dda5223466d58e55dc8392
Jul 16 21:16:28.624136 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:28.624058    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2r98p" event={"ID":"72a05823-0e7c-4d2b-b6be-2a6972a2827d","Type":"ContainerStarted","Data":"f3dfaf92ee0cf1660a2a94ea4ce3368b1376f740e398fde76b2dac8d72eb94a7"}
Jul 16 21:16:28.625969 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:28.625934    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdb985c9c-qm9ks" event={"ID":"6fd5a03c-a188-4505-8f05-33bdd49300a6","Type":"ContainerStarted","Data":"bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c"}
Jul 16 21:16:28.627196 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:28.627170    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf" event={"ID":"183b5c61-663b-423c-8b1b-e3c07131cb74","Type":"ContainerStarted","Data":"ffdd65a41c839c719946a8b3a3bbac686d7e7c96e4dda5223466d58e55dc8392"}
Jul 16 21:16:28.657630 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:28.657569    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fdb985c9c-qm9ks" podStartSLOduration=6.813209496 podStartE2EDuration="12.657547483s" totalImagesPullingTime="5.844337987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:28.656866139 +0000 UTC m=+54.168025874" watchObservedRunningTime="2026-07-16 21:16:28.657547483 +0000 UTC m=+54.168707209"
Jul 16 21:16:30.648381 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:30.648321    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf" event={"ID":"183b5c61-663b-423c-8b1b-e3c07131cb74","Type":"ContainerStarted","Data":"3a5bd981b13ab0c85b1d08efb8b3dbd503653de09f0b3c18935b9db15e3ffd9d"}
Jul 16 21:16:30.648840 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:30.648446    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf" event={"ID":"183b5c61-663b-423c-8b1b-e3c07131cb74","Type":"ContainerStarted","Data":"a4a40ad25b16df08d64783f942814a531341020447b831e4afcc201860b86dd9"}
Jul 16 21:16:32.611211 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.611180    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pc7xc"
Jul 16 21:16:32.658598 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.658565    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2r98p" event={"ID":"72a05823-0e7c-4d2b-b6be-2a6972a2827d","Type":"ContainerStarted","Data":"94ff9107d88a85e6c8739a07a7223146085e5d39c629a38d5c499b3d2235ef83"}
Jul 16 21:16:32.669966 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.669908    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-bx9qf" podStartSLOduration=7.704715183 podStartE2EDuration="9.669891399s" totalImagesPullingTime="1.965176216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:30.681719471 +0000 UTC m=+56.192879206" watchObservedRunningTime="2026-07-16 21:16:32.669891399 +0000 UTC m=+58.181051133"
Jul 16 21:16:32.903741 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.903636    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2r98p" podStartSLOduration=1.897887566 podStartE2EDuration="18.90361628s" totalImagesPullingTime="17.005728714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:16:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:32.720669619 +0000 UTC m=+58.231829354" watchObservedRunningTime="2026-07-16 21:16:32.90361628 +0000 UTC m=+58.414776013"
Jul 16 21:16:32.903938 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.903910    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"]
Jul 16 21:16:32.906256 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.906239    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"]
Jul 16 21:16:32.906393 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.906377    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:32.908435 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.908415    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:32.922789 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.922761    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcvvt\" (UniqueName: \"kubernetes.io/projected/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-api-access-fcvvt\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:32.922918 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.922793    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5z7n\" (UniqueName: \"kubernetes.io/projected/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-kube-api-access-v5z7n\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:32.922918 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.922812    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:32.922918 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.922829    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:32.922918 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.922887    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:32.922918 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.922912    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:32.923081 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.922992    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:32.923081 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.923026    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:32.923081 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.923058    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:32.923169 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.923078    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:32.947195 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:32.947155    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-dockercfg-q9h97\" is forbidden: User \"system:node:ip-10-0-133-67.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q9h97\"" type="*v1.Secret"
Jul 16 21:16:32.947195 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:32.947153    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-state-metrics-56c5594674-qjfdp\" is forbidden: User \"system:node:ip-10-0-133-67.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object" podUID="d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002" pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:32.947395 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.947271    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 16 21:16:32.947395 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:32.947155    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-state-metrics-tls\" is forbidden: User \"system:node:ip-10-0-133-67.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" type="*v1.Secret"
Jul 16 21:16:32.951751 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.951722    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nrk4v"]
Jul 16 21:16:32.954179 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.954163    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:32.960312 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.960290    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 16 21:16:32.960716 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.960697    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 16 21:16:32.960930 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.960915    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-cz8b5\""
Jul 16 21:16:32.982761 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:32.982735    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 16 21:16:33.004174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.004136    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"]
Jul 16 21:16:33.023785 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023748    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.023949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023796    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.023949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023827    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-root\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.023949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023850    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-tls\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.023949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023874    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.023949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023907    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.023949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.024186 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023959    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-sys\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.024186 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.023997    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5468131-84f7-47ef-be17-47499500f8a6-metrics-client-ca\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.024186 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024053    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-wtmp\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.024186 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024094    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krhg7\" (UniqueName: \"kubernetes.io/projected/d5468131-84f7-47ef-be17-47499500f8a6-kube-api-access-krhg7\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.024186 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024131    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcvvt\" (UniqueName: \"kubernetes.io/projected/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-api-access-fcvvt\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.024186 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024157    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5z7n\" (UniqueName: \"kubernetes.io/projected/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-kube-api-access-v5z7n\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.024489 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024190    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.024489 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024252    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.024489 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024386    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.024489 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024472    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-textfile\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.024721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024505    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.024721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024538    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.024721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024567    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.024721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024578    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.024861 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.024837    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.025225 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.025198    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.026584 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.026564    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.026764 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.026743    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.026807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.026743    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.035865 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.035845    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 21:16:33.038878 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.038863    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 21:16:33.039102 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.039089    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pnkrk\""
Jul 16 21:16:33.039395 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.039380    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 21:16:33.055176 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.055146    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"]
Jul 16 21:16:33.090797 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.090758    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5z7n\" (UniqueName: \"kubernetes.io/projected/9dec2bf2-90be-4d7c-9263-7ec2db3f17bd-kube-api-access-v5z7n\") pod \"openshift-state-metrics-79f6958cdf-26pzx\" (UID: \"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.092212 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.092175    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcvvt\" (UniqueName: \"kubernetes.io/projected/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-api-access-fcvvt\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:33.125065 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125029    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125075    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-root\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125094    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-tls\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125113    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125142    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-sys\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125158    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-root\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125171    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5468131-84f7-47ef-be17-47499500f8a6-metrics-client-ca\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125226    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-wtmp\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125258    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krhg7\" (UniqueName: \"kubernetes.io/projected/d5468131-84f7-47ef-be17-47499500f8a6-kube-api-access-krhg7\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125267    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-sys\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125338    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-textfile\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125672 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125453    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-wtmp\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125680    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-textfile\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125792    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5468131-84f7-47ef-be17-47499500f8a6-metrics-client-ca\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.125906 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.125847    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-accelerators-collector-config\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.127537 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.127516    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.127682 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.127662    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d5468131-84f7-47ef-be17-47499500f8a6-node-exporter-tls\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.156333 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.156246    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krhg7\" (UniqueName: \"kubernetes.io/projected/d5468131-84f7-47ef-be17-47499500f8a6-kube-api-access-krhg7\") pod \"node-exporter-nrk4v\" (UID: \"d5468131-84f7-47ef-be17-47499500f8a6\") " pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.221793 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.221750    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"
Jul 16 21:16:33.262898 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.262866    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nrk4v"
Jul 16 21:16:33.273275 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:33.273241    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5468131_84f7_47ef_be17_47499500f8a6.slice/crio-b0590718bcdefa17d05a57172fcb9f0c8ed44d016546460dd7d8f8d5a3a48f3e WatchSource:0}: Error finding container b0590718bcdefa17d05a57172fcb9f0c8ed44d016546460dd7d8f8d5a3a48f3e: Status 404 returned error can't find the container with id b0590718bcdefa17d05a57172fcb9f0c8ed44d016546460dd7d8f8d5a3a48f3e
Jul 16 21:16:33.375197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.375165    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx"]
Jul 16 21:16:33.378536 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:33.378504    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dec2bf2_90be_4d7c_9263_7ec2db3f17bd.slice/crio-0b09ebbd3b1a8420a70ddd8563e43859fd07f07f97a9cface7ff7d179552e676 WatchSource:0}: Error finding container 0b09ebbd3b1a8420a70ddd8563e43859fd07f07f97a9cface7ff7d179552e676: Status 404 returned error can't find the container with id 0b09ebbd3b1a8420a70ddd8563e43859fd07f07f97a9cface7ff7d179552e676
Jul 16 21:16:33.664195 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.664109    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nrk4v" event={"ID":"d5468131-84f7-47ef-be17-47499500f8a6","Type":"ContainerStarted","Data":"b0590718bcdefa17d05a57172fcb9f0c8ed44d016546460dd7d8f8d5a3a48f3e"}
Jul 16 21:16:33.666256 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.666220    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx" event={"ID":"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd","Type":"ContainerStarted","Data":"ffbd15e87c535ccac2f498569bb7f092866e8d07677d51da5e7474cd50aca2bc"}
Jul 16 21:16:33.666393 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.666280    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx" event={"ID":"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd","Type":"ContainerStarted","Data":"7690e050c7ffa5b7e4c07ad0a469fd10e0d3383a4b1a4d5beb959d2c826eda29"}
Jul 16 21:16:33.666393 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.666302    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx" event={"ID":"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd","Type":"ContainerStarted","Data":"0b09ebbd3b1a8420a70ddd8563e43859fd07f07f97a9cface7ff7d179552e676"}
Jul 16 21:16:33.888129 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:33.888096    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 16 21:16:33.895015 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:33.894979    2572 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 16 21:16:33.895193 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:16:33.895078    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-tls podName:d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002 nodeName:}" failed. No retries permitted until 2026-07-16 21:16:34.395055786 +0000 UTC m=+59.906215517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-qjfdp" (UID: "d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002") : secret "kube-state-metrics-tls" not found
Jul 16 21:16:34.439061 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:34.439027    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:34.441895 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:34.441840    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-qjfdp\" (UID: \"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:34.548868 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:34.548829    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q9h97\""
Jul 16 21:16:34.672125 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:34.672088    2572 generic.go:356] "Generic (PLEG): container finished" podID="d5468131-84f7-47ef-be17-47499500f8a6" containerID="147e013efc421bedddf9277108e8ec861b5d09dd011c7a6b167c5f8bc1575167" exitCode=0
Jul 16 21:16:34.672592 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:34.672164    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nrk4v" event={"ID":"d5468131-84f7-47ef-be17-47499500f8a6","Type":"ContainerDied","Data":"147e013efc421bedddf9277108e8ec861b5d09dd011c7a6b167c5f8bc1575167"}
Jul 16 21:16:34.715698 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:34.715667    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"
Jul 16 21:16:34.748243 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:34.748197    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx" podStartSLOduration=1.6416282309999999 podStartE2EDuration="2.748178397s" totalImagesPullingTime="1.106550166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:34.747576029 +0000 UTC m=+60.258735765" watchObservedRunningTime="2026-07-16 21:16:34.748178397 +0000 UTC m=+60.259338130"
Jul 16 21:16:34.873883 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:34.873806    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-qjfdp"]
Jul 16 21:16:34.876738 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:34.876710    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60cacca_9aa4_4c3d_b4ab_4c34c2dc3002.slice/crio-39f5363b36681421f7e2f0b278aa9840660aa3f81e999382589484067e1d3172 WatchSource:0}: Error finding container 39f5363b36681421f7e2f0b278aa9840660aa3f81e999382589484067e1d3172: Status 404 returned error can't find the container with id 39f5363b36681421f7e2f0b278aa9840660aa3f81e999382589484067e1d3172
Jul 16 21:16:35.544185 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:35.544155    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vstcz"
Jul 16 21:16:35.680638 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:35.680596    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp" event={"ID":"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002","Type":"ContainerStarted","Data":"39f5363b36681421f7e2f0b278aa9840660aa3f81e999382589484067e1d3172"}
Jul 16 21:16:35.687092 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:35.687048    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nrk4v" event={"ID":"d5468131-84f7-47ef-be17-47499500f8a6","Type":"ContainerStarted","Data":"28283ef8b4da55c33b2fef1d9ed53c88b3e0e5da1c3e2661569cde7380ca2342"}
Jul 16 21:16:35.687262 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:35.687153    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nrk4v" event={"ID":"d5468131-84f7-47ef-be17-47499500f8a6","Type":"ContainerStarted","Data":"12e0d2de209c1b0e0669af9de4b2c706a4a73f3f2acccf81b4a39d4918c64e05"}
Jul 16 21:16:35.689667 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:35.689639    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-26pzx" event={"ID":"9dec2bf2-90be-4d7c-9263-7ec2db3f17bd","Type":"ContainerStarted","Data":"8399814b7a2f1e90cfabfd3b665625265888616d68c6cd200daf63c118c789a8"}
Jul 16 21:16:35.761337 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:35.761285    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nrk4v" podStartSLOduration=2.656988647 podStartE2EDuration="3.761266836s" totalImagesPullingTime="1.028733621s" totalInitContainerRuntime="75.544568ms" isStatefulPod=true podCreationTimestamp="2026-07-16 21:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:35.759961545 +0000 UTC m=+61.271121291" watchObservedRunningTime="2026-07-16 21:16:35.761266836 +0000 UTC m=+61.272426571"
Jul 16 21:16:36.840881 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:36.840840    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:36.841320 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:36.840910    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:36.845905 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:36.845879    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:37.438135 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.438055    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6587bf557-82qg4"]
Jul 16 21:16:37.440809 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.440783    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.444623 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.444601    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 16 21:16:37.444749 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.444704    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 16 21:16:37.448932 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.448913    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 16 21:16:37.460895 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.460871    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-w52zs\""
Jul 16 21:16:37.461040 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.460899    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 16 21:16:37.461040 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.461027    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 16 21:16:37.461474 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.461447    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-eeoa3kaeclnto\""
Jul 16 21:16:37.462521 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.462499    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.462604 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.462543    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.462604 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.462577    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-grpc-tls\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.462604 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.462600    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-tls\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.462701 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.462638    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f4701a8-47d7-4d26-adfa-1f748101d1d9-metrics-client-ca\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.462701 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.462662    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.462701 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.462686    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szvb2\" (UniqueName: \"kubernetes.io/projected/9f4701a8-47d7-4d26-adfa-1f748101d1d9-kube-api-access-szvb2\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.462791 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.462709    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.563669 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.563631    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f4701a8-47d7-4d26-adfa-1f748101d1d9-metrics-client-ca\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.563867 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.563679    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.563867 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.563719    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szvb2\" (UniqueName: \"kubernetes.io/projected/9f4701a8-47d7-4d26-adfa-1f748101d1d9-kube-api-access-szvb2\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.563867 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.563747    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.563867 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.563797    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.563867 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.563845    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.564122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.563881    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-grpc-tls\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.564122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.563913    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-tls\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.564612 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.564558    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f4701a8-47d7-4d26-adfa-1f748101d1d9-metrics-client-ca\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.566706 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.566678    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.567035 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.566983    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.567142 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.567034    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.567197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.567148    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-tls\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.567197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.567160    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.567678 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.567656    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9f4701a8-47d7-4d26-adfa-1f748101d1d9-secret-grpc-tls\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.594332 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.594300    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szvb2\" (UniqueName: \"kubernetes.io/projected/9f4701a8-47d7-4d26-adfa-1f748101d1d9-kube-api-access-szvb2\") pod \"thanos-querier-6587bf557-82qg4\" (UID: \"9f4701a8-47d7-4d26-adfa-1f748101d1d9\") " pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.694082 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.693944    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6587bf557-82qg4"]
Jul 16 21:16:37.698949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.698906    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp" event={"ID":"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002","Type":"ContainerStarted","Data":"6671f3facfdb2e54ed156d6ef9c0e06ea5a45c73ec4f74d142b16893c8705e82"}
Jul 16 21:16:37.699126 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.698982    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp" event={"ID":"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002","Type":"ContainerStarted","Data":"c8cccc2b722b2322e94e6b5f4223c97140e8b43842ea6807e4eb53ead5560607"}
Jul 16 21:16:37.699126 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.699024    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp" event={"ID":"d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002","Type":"ContainerStarted","Data":"bf385517fa6750e3d65f423a9b88819e9f9a86c7a9113a6028da23352be9f839"}
Jul 16 21:16:37.703060 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.703026    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:16:37.742979 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.742926    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-qjfdp" podStartSLOduration=3.698242615 podStartE2EDuration="5.742908908s" totalImagesPullingTime="2.044666293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:37.739208361 +0000 UTC m=+63.250368096" watchObservedRunningTime="2026-07-16 21:16:37.742908908 +0000 UTC m=+63.254068642"
Jul 16 21:16:37.750595 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.750565    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:37.996626 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:37.996598    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f4701a8_47d7_4d26_adfa_1f748101d1d9.slice/crio-1b6d4f74ebbeb1ea56d890e26f683767bf7d9b03392f2142a168553ee92442b1 WatchSource:0}: Error finding container 1b6d4f74ebbeb1ea56d890e26f683767bf7d9b03392f2142a168553ee92442b1: Status 404 returned error can't find the container with id 1b6d4f74ebbeb1ea56d890e26f683767bf7d9b03392f2142a168553ee92442b1
Jul 16 21:16:37.997861 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:37.997829    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6587bf557-82qg4"]
Jul 16 21:16:38.112453 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.112417    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-64cd49d94d-rk7lx"]
Jul 16 21:16:38.117767 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.116596    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.120850 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.120807    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 16 21:16:38.120981 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.120856    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fl6ibfh42at9j\""
Jul 16 21:16:38.120981 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.120952    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 16 21:16:38.121367 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.121333    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-8rhfg\""
Jul 16 21:16:38.121478 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.121401    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 21:16:38.122014 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.121993    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 16 21:16:38.151827 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.151795    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64cd49d94d-rk7lx"]
Jul 16 21:16:38.170531 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.170495    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2hv7\" (UniqueName: \"kubernetes.io/projected/5996081f-4f04-4cf0-819f-ab4e356b2163-kube-api-access-j2hv7\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.170531 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.170533    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-secret-metrics-server-tls\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.170737 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.170553    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5996081f-4f04-4cf0-819f-ab4e356b2163-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.170737 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.170614    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5996081f-4f04-4cf0-819f-ab4e356b2163-audit-log\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.170737 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.170693    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-client-ca-bundle\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.170737 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.170729    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5996081f-4f04-4cf0-819f-ab4e356b2163-metrics-server-audit-profiles\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.170875 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.170756    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-secret-metrics-server-client-certs\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.271891 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.271788    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5996081f-4f04-4cf0-819f-ab4e356b2163-audit-log\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.271891 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.271862    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-client-ca-bundle\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.272109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.271897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5996081f-4f04-4cf0-819f-ab4e356b2163-metrics-server-audit-profiles\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.272109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.271916    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-secret-metrics-server-client-certs\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.272214 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.272110    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2hv7\" (UniqueName: \"kubernetes.io/projected/5996081f-4f04-4cf0-819f-ab4e356b2163-kube-api-access-j2hv7\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.272214 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.272153    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-secret-metrics-server-tls\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.272214 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.272182    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5996081f-4f04-4cf0-819f-ab4e356b2163-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.272390 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.272265    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5996081f-4f04-4cf0-819f-ab4e356b2163-audit-log\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.272807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.272786    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5996081f-4f04-4cf0-819f-ab4e356b2163-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.272970 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.272950    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5996081f-4f04-4cf0-819f-ab4e356b2163-metrics-server-audit-profiles\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.275864 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.274812    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-secret-metrics-server-client-certs\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.275864 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.274916    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-client-ca-bundle\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.276577 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.276554    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5996081f-4f04-4cf0-819f-ab4e356b2163-secret-metrics-server-tls\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.294911 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.294879    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2hv7\" (UniqueName: \"kubernetes.io/projected/5996081f-4f04-4cf0-819f-ab4e356b2163-kube-api-access-j2hv7\") pod \"metrics-server-64cd49d94d-rk7lx\" (UID: \"5996081f-4f04-4cf0-819f-ab4e356b2163\") " pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.427455 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.427406    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:38.598215 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.598186    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64cd49d94d-rk7lx"]
Jul 16 21:16:38.600832 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:38.600804    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5996081f_4f04_4cf0_819f_ab4e356b2163.slice/crio-30de217f77cf8631a85ec0beae0a9ced8c2633e8f8812da69a23fd8f80aea65d WatchSource:0}: Error finding container 30de217f77cf8631a85ec0beae0a9ced8c2633e8f8812da69a23fd8f80aea65d: Status 404 returned error can't find the container with id 30de217f77cf8631a85ec0beae0a9ced8c2633e8f8812da69a23fd8f80aea65d
Jul 16 21:16:38.704774 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.704731    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4" event={"ID":"9f4701a8-47d7-4d26-adfa-1f748101d1d9","Type":"ContainerStarted","Data":"1b6d4f74ebbeb1ea56d890e26f683767bf7d9b03392f2142a168553ee92442b1"}
Jul 16 21:16:38.706163 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:38.706136    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx" event={"ID":"5996081f-4f04-4cf0-819f-ab4e356b2163","Type":"ContainerStarted","Data":"30de217f77cf8631a85ec0beae0a9ced8c2633e8f8812da69a23fd8f80aea65d"}
Jul 16 21:16:40.230232 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.230194    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d69fc5976-xlds9"]
Jul 16 21:16:40.232834 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.232807    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.276378 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.276334    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d69fc5976-xlds9"]
Jul 16 21:16:40.293055 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.292854    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-trusted-ca-bundle\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.293055 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.292929    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-serving-cert\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.293055 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.292991    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-oauth-config\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.293055 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.293023    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-config\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.293420 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.293092    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-service-ca\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.293420 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.293179    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-oauth-serving-cert\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.293420 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.293202    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7plnp\" (UniqueName: \"kubernetes.io/projected/60773ed6-ab66-4e96-8e91-ad608e07a6f6-kube-api-access-7plnp\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.394520 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.394482    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-trusted-ca-bundle\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.394722 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.394556    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-serving-cert\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.394722 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.394595    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-oauth-config\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.394722 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.394627    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-config\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.394722 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.394655    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-service-ca\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.394722 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.394713    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-oauth-serving-cert\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.394969 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.394736    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7plnp\" (UniqueName: \"kubernetes.io/projected/60773ed6-ab66-4e96-8e91-ad608e07a6f6-kube-api-access-7plnp\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.395500 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.395471    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-trusted-ca-bundle\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.395628 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.395573    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-config\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.395628 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.395576    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-service-ca\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.395792 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.395762    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-oauth-serving-cert\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.397605 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.397579    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-oauth-config\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.397740 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.397717    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-serving-cert\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.422936 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.422866    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7plnp\" (UniqueName: \"kubernetes.io/projected/60773ed6-ab66-4e96-8e91-ad608e07a6f6-kube-api-access-7plnp\") pod \"console-6d69fc5976-xlds9\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") " pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.544681 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.544648    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:40.718980 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.718136    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx" event={"ID":"5996081f-4f04-4cf0-819f-ab4e356b2163","Type":"ContainerStarted","Data":"86d842884ba44c0b805b662a176769d564a362f75903aab141797965c6fb0e3b"}
Jul 16 21:16:40.722917 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.722684    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4" event={"ID":"9f4701a8-47d7-4d26-adfa-1f748101d1d9","Type":"ContainerStarted","Data":"13b56abf110dc920d427b1a543d8b28515d35e33587c1de3bca2c754d0f67043"}
Jul 16 21:16:40.802967 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.802911    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx" podStartSLOduration=0.806589771 podStartE2EDuration="2.802889311s" totalImagesPullingTime="1.99629954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:40.777587474 +0000 UTC m=+66.288747210" watchObservedRunningTime="2026-07-16 21:16:40.802889311 +0000 UTC m=+66.314049026"
Jul 16 21:16:40.804064 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.804043    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d69fc5976-xlds9"]
Jul 16 21:16:40.807281 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:40.807252    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60773ed6_ab66_4e96_8e91_ad608e07a6f6.slice/crio-d8986ac9c4423374c146875052ab04342e933576ccdd8f01a308b7f998923f8c WatchSource:0}: Error finding container d8986ac9c4423374c146875052ab04342e933576ccdd8f01a308b7f998923f8c: Status 404 returned error can't find the container with id d8986ac9c4423374c146875052ab04342e933576ccdd8f01a308b7f998923f8c
Jul 16 21:16:40.999569 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.999525    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:40.999742 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:40.999629    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:41.012498 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.012463    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 21:16:41.012652 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.012580    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 21:16:41.022612 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.022579    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/885f0bbb-17f9-4438-b179-2d716e4cc295-metrics-certs\") pod \"network-metrics-daemon-2wwbb\" (UID: \"885f0bbb-17f9-4438-b179-2d716e4cc295\") " pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:41.022612 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.022599    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/61292e97-d6af-40aa-9f6b-c944ebe621b3-original-pull-secret\") pod \"global-pull-secret-syncer-zjlgl\" (UID: \"61292e97-d6af-40aa-9f6b-c944ebe621b3\") " pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:41.065035 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.064954    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tpjps\""
Jul 16 21:16:41.072188 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.072161    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2wwbb"
Jul 16 21:16:41.100604 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.100572    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:41.106950 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.106926    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 21:16:41.124688 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.124653    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 21:16:41.135477 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.135417    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm58n\" (UniqueName: \"kubernetes.io/projected/cc75c925-0a50-440b-8def-b991e61e560e-kube-api-access-tm58n\") pod \"network-check-target-ddb52\" (UID: \"cc75c925-0a50-440b-8def-b991e61e560e\") " pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:41.157261 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.157211    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zjlgl"
Jul 16 21:16:41.257057 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.256924    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2wwbb"]
Jul 16 21:16:41.261874 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:41.261848    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885f0bbb_17f9_4438_b179_2d716e4cc295.slice/crio-068a70e4d3ae93d0af0eec2736fc4934fc76ccc3961c61b1dc5b14a70df71d69 WatchSource:0}: Error finding container 068a70e4d3ae93d0af0eec2736fc4934fc76ccc3961c61b1dc5b14a70df71d69: Status 404 returned error can't find the container with id 068a70e4d3ae93d0af0eec2736fc4934fc76ccc3961c61b1dc5b14a70df71d69
Jul 16 21:16:41.354876 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.354797    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zjlgl"]
Jul 16 21:16:41.359989 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.359964    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzz4k\""
Jul 16 21:16:41.366056 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.366028    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:41.402338 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:41.402297    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61292e97_d6af_40aa_9f6b_c944ebe621b3.slice/crio-aac1da93a74bf47cfc4872e780945d4e46fcbe3ae01c1c42f424fa9ddf0f8cd0 WatchSource:0}: Error finding container aac1da93a74bf47cfc4872e780945d4e46fcbe3ae01c1c42f424fa9ddf0f8cd0: Status 404 returned error can't find the container with id aac1da93a74bf47cfc4872e780945d4e46fcbe3ae01c1c42f424fa9ddf0f8cd0
Jul 16 21:16:41.672810 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.672782    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ddb52"]
Jul 16 21:16:41.672938 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:16:41.672777    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc75c925_0a50_440b_8def_b991e61e560e.slice/crio-6e7501576b21d0fadb2d48f71a5867618e1bf17ede8c92b72d5994cc8a7dba53 WatchSource:0}: Error finding container 6e7501576b21d0fadb2d48f71a5867618e1bf17ede8c92b72d5994cc8a7dba53: Status 404 returned error can't find the container with id 6e7501576b21d0fadb2d48f71a5867618e1bf17ede8c92b72d5994cc8a7dba53
Jul 16 21:16:41.728145 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.728104    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wwbb" event={"ID":"885f0bbb-17f9-4438-b179-2d716e4cc295","Type":"ContainerStarted","Data":"068a70e4d3ae93d0af0eec2736fc4934fc76ccc3961c61b1dc5b14a70df71d69"}
Jul 16 21:16:41.729332 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.729307    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ddb52" event={"ID":"cc75c925-0a50-440b-8def-b991e61e560e","Type":"ContainerStarted","Data":"6e7501576b21d0fadb2d48f71a5867618e1bf17ede8c92b72d5994cc8a7dba53"}
Jul 16 21:16:41.730304 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.730281    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zjlgl" event={"ID":"61292e97-d6af-40aa-9f6b-c944ebe621b3","Type":"ContainerStarted","Data":"aac1da93a74bf47cfc4872e780945d4e46fcbe3ae01c1c42f424fa9ddf0f8cd0"}
Jul 16 21:16:41.731895 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.731872    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d69fc5976-xlds9" event={"ID":"60773ed6-ab66-4e96-8e91-ad608e07a6f6","Type":"ContainerStarted","Data":"b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7"}
Jul 16 21:16:41.731968 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.731906    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d69fc5976-xlds9" event={"ID":"60773ed6-ab66-4e96-8e91-ad608e07a6f6","Type":"ContainerStarted","Data":"d8986ac9c4423374c146875052ab04342e933576ccdd8f01a308b7f998923f8c"}
Jul 16 21:16:41.734753 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.734722    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4" event={"ID":"9f4701a8-47d7-4d26-adfa-1f748101d1d9","Type":"ContainerStarted","Data":"b43fbaaa9270bb373f091524018a05560b23271d23a5dadde6023d33b135b7bd"}
Jul 16 21:16:41.734861 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.734782    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4" event={"ID":"9f4701a8-47d7-4d26-adfa-1f748101d1d9","Type":"ContainerStarted","Data":"8fc990fe0f4c435a195d378d442c47742723b8065154d6e7961cf7bbc5406d24"}
Jul 16 21:16:41.734861 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.734809    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4" event={"ID":"9f4701a8-47d7-4d26-adfa-1f748101d1d9","Type":"ContainerStarted","Data":"51ec28ab0ffdda5f9c8e72f2fa643d38c2f6992eaa9b6afbc61f399bd1cf76de"}
Jul 16 21:16:41.734861 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.734832    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4" event={"ID":"9f4701a8-47d7-4d26-adfa-1f748101d1d9","Type":"ContainerStarted","Data":"8644f2d1f9f85cb4edb2b355785d784be56f9ce82c39504d5818b922c6742c17"}
Jul 16 21:16:41.734861 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.734859    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4" event={"ID":"9f4701a8-47d7-4d26-adfa-1f748101d1d9","Type":"ContainerStarted","Data":"3c44591e22af1c93a7dde2e800f693eada9cfcccaba379885d45fe26f0f921a0"}
Jul 16 21:16:41.888752 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.888651    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d69fc5976-xlds9" podStartSLOduration=1.888631185 podStartE2EDuration="1.888631185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:41.88759796 +0000 UTC m=+67.398757695" watchObservedRunningTime="2026-07-16 21:16:41.888631185 +0000 UTC m=+67.399790926"
Jul 16 21:16:41.931096 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:41.931042    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4" podStartSLOduration=1.7073721960000001 podStartE2EDuration="4.931024787s" totalImagesPullingTime="3.223652591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:41.930961619 +0000 UTC m=+67.442121352" watchObservedRunningTime="2026-07-16 21:16:41.931024787 +0000 UTC m=+67.442184499"
Jul 16 21:16:42.740492 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:42.740453    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:43.747322 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:43.747277    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wwbb" event={"ID":"885f0bbb-17f9-4438-b179-2d716e4cc295","Type":"ContainerStarted","Data":"d01fe59b4b3d4825f35e7ea367675e14927c48d72078c38b48f417c88d83e2da"}
Jul 16 21:16:43.747874 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:43.747420    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2wwbb" event={"ID":"885f0bbb-17f9-4438-b179-2d716e4cc295","Type":"ContainerStarted","Data":"8fbe7da94facdd4f09a21e4a84d8b4a95c1dadb7146685a784e560fe769eb2dc"}
Jul 16 21:16:43.810713 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:43.810646    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2wwbb" podStartSLOduration=67.10470817 podStartE2EDuration="1m8.810622716s" totalImagesPullingTime="1.705914546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:43.808905645 +0000 UTC m=+69.320065401" watchObservedRunningTime="2026-07-16 21:16:43.810622716 +0000 UTC m=+69.321782452"
Jul 16 21:16:47.765664 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:47.765626    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ddb52" event={"ID":"cc75c925-0a50-440b-8def-b991e61e560e","Type":"ContainerStarted","Data":"ff3e1b8ac6868475009bae474761b22b83b341e2ccb3ae61856c067725d5e0e9"}
Jul 16 21:16:47.766101 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:47.765714    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:16:47.767087 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:47.767065    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zjlgl" event={"ID":"61292e97-d6af-40aa-9f6b-c944ebe621b3","Type":"ContainerStarted","Data":"8b5f64cca39e17fe9526aba781069274f937af535fa4d53910014ac1433c37ac"}
Jul 16 21:16:47.807160 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:47.807105    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ddb52" podStartSLOduration=67.71794471 podStartE2EDuration="1m12.807090462s" totalImagesPullingTime="5.089145752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:47.805342491 +0000 UTC m=+73.316502226" watchObservedRunningTime="2026-07-16 21:16:47.807090462 +0000 UTC m=+73.318250240"
Jul 16 21:16:47.858817 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:47.858752    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zjlgl" podStartSLOduration=67.49585458 podStartE2EDuration="1m12.858729926s" totalImagesPullingTime="5.362875346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:47.857708201 +0000 UTC m=+73.368867938" watchObservedRunningTime="2026-07-16 21:16:47.858729926 +0000 UTC m=+73.369889663"
Jul 16 21:16:48.756161 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:48.756132    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6587bf557-82qg4"
Jul 16 21:16:50.544788 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:50.544748    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:50.545181 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:50.544806    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:50.549541 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:50.549515    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:50.784503 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:50.784471    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:16:50.894683 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:50.894517    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fdb985c9c-qm9ks"]
Jul 16 21:16:58.428259 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:58.428137    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:16:58.428259 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:16:58.428208    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:17:15.917312 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:15.917264    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fdb985c9c-qm9ks" podUID="6fd5a03c-a188-4505-8f05-33bdd49300a6" containerName="console" containerID="cri-o://bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c" gracePeriod=15
Jul 16 21:17:16.161605 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.161581    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:17:16.194719 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.194636    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2btw2\" (UniqueName: \"kubernetes.io/projected/6fd5a03c-a188-4505-8f05-33bdd49300a6-kube-api-access-2btw2\") pod \"6fd5a03c-a188-4505-8f05-33bdd49300a6\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") "
Jul 16 21:17:16.194719 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.194676    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-config\") pod \"6fd5a03c-a188-4505-8f05-33bdd49300a6\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") "
Jul 16 21:17:16.194719 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.194715    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-trusted-ca-bundle\") pod \"6fd5a03c-a188-4505-8f05-33bdd49300a6\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") "
Jul 16 21:17:16.194972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.194741    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-oauth-config\") pod \"6fd5a03c-a188-4505-8f05-33bdd49300a6\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") "
Jul 16 21:17:16.194972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.194774    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-serving-cert\") pod \"6fd5a03c-a188-4505-8f05-33bdd49300a6\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") "
Jul 16 21:17:16.194972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.194821    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-oauth-serving-cert\") pod \"6fd5a03c-a188-4505-8f05-33bdd49300a6\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") "
Jul 16 21:17:16.194972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.194849    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-service-ca\") pod \"6fd5a03c-a188-4505-8f05-33bdd49300a6\" (UID: \"6fd5a03c-a188-4505-8f05-33bdd49300a6\") "
Jul 16 21:17:16.195168 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.195148    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-config" (OuterVolumeSpecName: "console-config") pod "6fd5a03c-a188-4505-8f05-33bdd49300a6" (UID: "6fd5a03c-a188-4505-8f05-33bdd49300a6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:16.195287 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.195255    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6fd5a03c-a188-4505-8f05-33bdd49300a6" (UID: "6fd5a03c-a188-4505-8f05-33bdd49300a6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:16.195374 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.195317    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-service-ca" (OuterVolumeSpecName: "service-ca") pod "6fd5a03c-a188-4505-8f05-33bdd49300a6" (UID: "6fd5a03c-a188-4505-8f05-33bdd49300a6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:16.195567 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.195537    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6fd5a03c-a188-4505-8f05-33bdd49300a6" (UID: "6fd5a03c-a188-4505-8f05-33bdd49300a6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:17:16.197117 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.197092    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6fd5a03c-a188-4505-8f05-33bdd49300a6" (UID: "6fd5a03c-a188-4505-8f05-33bdd49300a6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:16.197209 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.197185    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6fd5a03c-a188-4505-8f05-33bdd49300a6" (UID: "6fd5a03c-a188-4505-8f05-33bdd49300a6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:16.197251 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.197220    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd5a03c-a188-4505-8f05-33bdd49300a6-kube-api-access-2btw2" (OuterVolumeSpecName: "kube-api-access-2btw2") pod "6fd5a03c-a188-4505-8f05-33bdd49300a6" (UID: "6fd5a03c-a188-4505-8f05-33bdd49300a6"). InnerVolumeSpecName "kube-api-access-2btw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:17:16.295455 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.295414    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2btw2\" (UniqueName: \"kubernetes.io/projected/6fd5a03c-a188-4505-8f05-33bdd49300a6-kube-api-access-2btw2\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:17:16.295455 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.295450    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-config\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:17:16.295455 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.295461    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-trusted-ca-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:17:16.295696 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.295471    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-oauth-config\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:17:16.295696 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.295481    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6fd5a03c-a188-4505-8f05-33bdd49300a6-console-serving-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:17:16.295696 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.295489    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-oauth-serving-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:17:16.295696 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.295499    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6fd5a03c-a188-4505-8f05-33bdd49300a6-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:17:16.880502 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.880458    2572 generic.go:356] "Generic (PLEG): container finished" podID="6fd5a03c-a188-4505-8f05-33bdd49300a6" containerID="bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c" exitCode=0
Jul 16 21:17:16.880502 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.880505    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdb985c9c-qm9ks"
Jul 16 21:17:16.880715 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.880549    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdb985c9c-qm9ks" event={"ID":"6fd5a03c-a188-4505-8f05-33bdd49300a6","Type":"ContainerDied","Data":"bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c"}
Jul 16 21:17:16.880715 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.880602    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdb985c9c-qm9ks" event={"ID":"6fd5a03c-a188-4505-8f05-33bdd49300a6","Type":"ContainerDied","Data":"f2bba753c5fda89bdd7738271a93b6f512b314858d1ff96b91d87139e6f4adcd"}
Jul 16 21:17:16.880715 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.880626    2572 scope.go:117] "RemoveContainer" containerID="bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c"
Jul 16 21:17:16.892809 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.892787    2572 scope.go:117] "RemoveContainer" containerID="bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c"
Jul 16 21:17:16.893101 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:17:16.893080    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c\": container with ID starting with bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c not found: ID does not exist" containerID="bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c"
Jul 16 21:17:16.893149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.893109    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c"} err="failed to get container status \"bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c\": rpc error: code = NotFound desc = could not find container \"bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c\": container with ID starting with bdf7fd163a26c8d820321199bcebf2c258862bd73d1033670ec7de4de52df18c not found: ID does not exist"
Jul 16 21:17:16.915330 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.915295    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fdb985c9c-qm9ks"]
Jul 16 21:17:16.924234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:16.924208    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fdb985c9c-qm9ks"]
Jul 16 21:17:17.248359 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:17.248260    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd5a03c-a188-4505-8f05-33bdd49300a6" path="/var/lib/kubelet/pods/6fd5a03c-a188-4505-8f05-33bdd49300a6/volumes"
Jul 16 21:17:18.433512 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:18.433479    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:17:18.437247 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:18.437224    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-64cd49d94d-rk7lx"
Jul 16 21:17:18.774184 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:18.774160    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ddb52"
Jul 16 21:17:57.824819 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.824782    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-59f57d446c-l9rfc"]
Jul 16 21:17:57.825268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.825056    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd5a03c-a188-4505-8f05-33bdd49300a6" containerName="console"
Jul 16 21:17:57.825268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.825067    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd5a03c-a188-4505-8f05-33bdd49300a6" containerName="console"
Jul 16 21:17:57.825268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.825114    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd5a03c-a188-4505-8f05-33bdd49300a6" containerName="console"
Jul 16 21:17:57.826979 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.826964    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:57.829724 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.829693    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 16 21:17:57.829904 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.829888    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 16 21:17:57.830278 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.830258    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-btss2\""
Jul 16 21:17:57.831042 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.831021    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 16 21:17:57.831147 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.831044    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 16 21:17:57.831288 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.831270    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 16 21:17:57.837307 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.837286    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 16 21:17:57.844735 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.844706    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-59f57d446c-l9rfc"]
Jul 16 21:17:57.900905 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.900869    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-telemeter-client-tls\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:57.900905 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.900906    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-metrics-client-ca\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:57.901143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.900944    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-secret-telemeter-client\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:57.901143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.900988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-serving-certs-ca-bundle\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:57.901143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.901028    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:57.901143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.901044    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcm9k\" (UniqueName: \"kubernetes.io/projected/8319506b-ee36-4500-8aa3-df405026e1bf-kube-api-access-lcm9k\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:57.901143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.901060    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-federate-client-tls\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:57.901143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:57.901078    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.002345 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.002295    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-secret-telemeter-client\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.002572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.002383    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-serving-certs-ca-bundle\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.002572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.002440    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.002572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.002466    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcm9k\" (UniqueName: \"kubernetes.io/projected/8319506b-ee36-4500-8aa3-df405026e1bf-kube-api-access-lcm9k\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.002572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.002491    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-federate-client-tls\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.002572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.002521    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.002572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.002557    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-telemeter-client-tls\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.002874 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.002582    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-metrics-client-ca\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.003401 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.003376    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-metrics-client-ca\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.004101 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.004078    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-serving-certs-ca-bundle\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.004568 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.004540    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8319506b-ee36-4500-8aa3-df405026e1bf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.006269 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.006233    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-federate-client-tls\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.006396 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.006321    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.006396 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.006340    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-secret-telemeter-client\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.006479 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.006396    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8319506b-ee36-4500-8aa3-df405026e1bf-telemeter-client-tls\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.012425 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.012325    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcm9k\" (UniqueName: \"kubernetes.io/projected/8319506b-ee36-4500-8aa3-df405026e1bf-kube-api-access-lcm9k\") pod \"telemeter-client-59f57d446c-l9rfc\" (UID: \"8319506b-ee36-4500-8aa3-df405026e1bf\") " pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.138414 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.138283    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc"
Jul 16 21:17:58.303040 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:58.303004    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-59f57d446c-l9rfc"]
Jul 16 21:17:58.307030 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:17:58.307003    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8319506b_ee36_4500_8aa3_df405026e1bf.slice/crio-ddab623e62a2b32f3e24f9b08e0811022d73ff7be212e3d66038f875541e670d WatchSource:0}: Error finding container ddab623e62a2b32f3e24f9b08e0811022d73ff7be212e3d66038f875541e670d: Status 404 returned error can't find the container with id ddab623e62a2b32f3e24f9b08e0811022d73ff7be212e3d66038f875541e670d
Jul 16 21:17:59.029582 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:17:59.029545    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc" event={"ID":"8319506b-ee36-4500-8aa3-df405026e1bf","Type":"ContainerStarted","Data":"ddab623e62a2b32f3e24f9b08e0811022d73ff7be212e3d66038f875541e670d"}
Jul 16 21:18:01.039236 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:01.039197    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc" event={"ID":"8319506b-ee36-4500-8aa3-df405026e1bf","Type":"ContainerStarted","Data":"09ff0ea369b9a6d217a1cd97b9d63348fc51028adc5e35333c5039f1ccfbeee8"}
Jul 16 21:18:03.049451 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:03.049409    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc" event={"ID":"8319506b-ee36-4500-8aa3-df405026e1bf","Type":"ContainerStarted","Data":"fd5bb8e0dd56be6d0dff932a84b71c5a10d02d5d6be920df546aebd8fa31dea4"}
Jul 16 21:18:03.049901 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:03.049466    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc" event={"ID":"8319506b-ee36-4500-8aa3-df405026e1bf","Type":"ContainerStarted","Data":"3f1b4e5743c8b2cc81cbf7e28ad8264a992017bb423df7836bd8f1b16345ddbb"}
Jul 16 21:18:03.965621 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:03.965557    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-59f57d446c-l9rfc" podStartSLOduration=3.23409438 podStartE2EDuration="6.965536408s" totalImagesPullingTime="3.731442028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:17:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:18:03.092411929 +0000 UTC m=+148.603571663" watchObservedRunningTime="2026-07-16 21:18:03.965536408 +0000 UTC m=+149.476696141"
Jul 16 21:18:03.966558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:03.966541    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6768c779bf-jdl42"]
Jul 16 21:18:03.968966 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:03.968952    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:03.977825 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:18:03.977787    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-6768c779bf-jdl42\" is forbidden: User \"system:node:ip-10-0-133-67.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object" podUID="6380c25c-646c-4eb8-9c06-d75a3a3e43d0" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.022507 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.022472    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6768c779bf-jdl42"]
Jul 16 21:18:04.049165 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.049131    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-trusted-ca-bundle\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.049165 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.049177    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-oauth-config\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.049411 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.049200    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-service-ca\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.049411 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.049218    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hscph\" (UniqueName: \"kubernetes.io/projected/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-kube-api-access-hscph\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.049411 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.049245    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-config\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.049411 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.049272    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-serving-cert\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.049411 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.049368    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-oauth-serving-cert\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.151323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.151273    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-oauth-serving-cert\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.151548 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.151380    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-trusted-ca-bundle\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.151548 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.151440    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-oauth-config\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.151548 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.151484    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-service-ca\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.151548 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.151507    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hscph\" (UniqueName: \"kubernetes.io/projected/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-kube-api-access-hscph\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.151548 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.151532    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-config\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.151798 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.151644    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-serving-cert\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.152328 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.152201    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-oauth-serving-cert\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.152328 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.152294    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-trusted-ca-bundle\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.152660 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.152631    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-service-ca\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.152771 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.152670    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-config\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.154619 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.154592    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-oauth-config\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.154699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.154601    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-serving-cert\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.169781 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.169747    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hscph\" (UniqueName: \"kubernetes.io/projected/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-kube-api-access-hscph\") pod \"console-6768c779bf-jdl42\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") " pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.279662 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.279622    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:04.443010 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:04.442984    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6768c779bf-jdl42"]
Jul 16 21:18:04.444722 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:18:04.444695    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6380c25c_646c_4eb8_9c06_d75a3a3e43d0.slice/crio-866d8c8b1de789c5e8ff3d674b63eb786db4047d1826002ce3139c4222ea3508 WatchSource:0}: Error finding container 866d8c8b1de789c5e8ff3d674b63eb786db4047d1826002ce3139c4222ea3508: Status 404 returned error can't find the container with id 866d8c8b1de789c5e8ff3d674b63eb786db4047d1826002ce3139c4222ea3508
Jul 16 21:18:05.058941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:05.058901    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6768c779bf-jdl42" event={"ID":"6380c25c-646c-4eb8-9c06-d75a3a3e43d0","Type":"ContainerStarted","Data":"f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157"}
Jul 16 21:18:05.059410 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:05.058952    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6768c779bf-jdl42" event={"ID":"6380c25c-646c-4eb8-9c06-d75a3a3e43d0","Type":"ContainerStarted","Data":"866d8c8b1de789c5e8ff3d674b63eb786db4047d1826002ce3139c4222ea3508"}
Jul 16 21:18:05.100152 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:05.100096    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6768c779bf-jdl42" podStartSLOduration=2.100080174 podStartE2EDuration="2.100080174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:18:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:18:05.098444767 +0000 UTC m=+150.609604501" watchObservedRunningTime="2026-07-16 21:18:05.100080174 +0000 UTC m=+150.611239908"
Jul 16 21:18:14.280662 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:14.280617    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:14.281183 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:14.280737    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:14.286740 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:14.286703    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:15.104453 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:15.104425    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:18:15.266239 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:15.266209    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d69fc5976-xlds9"]
Jul 16 21:18:40.285691 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.285581    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d69fc5976-xlds9" podUID="60773ed6-ab66-4e96-8e91-ad608e07a6f6" containerName="console" containerID="cri-o://b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7" gracePeriod=15
Jul 16 21:18:40.531877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.531853    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:18:40.641051 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.640951    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-trusted-ca-bundle\") pod \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") "
Jul 16 21:18:40.641051 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.640994    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-oauth-config\") pod \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") "
Jul 16 21:18:40.641051 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641024    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-serving-cert\") pod \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") "
Jul 16 21:18:40.641051 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641049    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-oauth-serving-cert\") pod \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") "
Jul 16 21:18:40.641394 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641087    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-service-ca\") pod \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") "
Jul 16 21:18:40.641394 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641134    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-config\") pod \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") "
Jul 16 21:18:40.641394 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641174    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7plnp\" (UniqueName: \"kubernetes.io/projected/60773ed6-ab66-4e96-8e91-ad608e07a6f6-kube-api-access-7plnp\") pod \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\" (UID: \"60773ed6-ab66-4e96-8e91-ad608e07a6f6\") "
Jul 16 21:18:40.641558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641503    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "60773ed6-ab66-4e96-8e91-ad608e07a6f6" (UID: "60773ed6-ab66-4e96-8e91-ad608e07a6f6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:18:40.641616 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641577    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-service-ca" (OuterVolumeSpecName: "service-ca") pod "60773ed6-ab66-4e96-8e91-ad608e07a6f6" (UID: "60773ed6-ab66-4e96-8e91-ad608e07a6f6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:18:40.641616 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641574    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-config" (OuterVolumeSpecName: "console-config") pod "60773ed6-ab66-4e96-8e91-ad608e07a6f6" (UID: "60773ed6-ab66-4e96-8e91-ad608e07a6f6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:18:40.641616 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.641580    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "60773ed6-ab66-4e96-8e91-ad608e07a6f6" (UID: "60773ed6-ab66-4e96-8e91-ad608e07a6f6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:18:40.643486 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.643449    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "60773ed6-ab66-4e96-8e91-ad608e07a6f6" (UID: "60773ed6-ab66-4e96-8e91-ad608e07a6f6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:18:40.643605 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.643540    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "60773ed6-ab66-4e96-8e91-ad608e07a6f6" (UID: "60773ed6-ab66-4e96-8e91-ad608e07a6f6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:18:40.643605 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.643561    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60773ed6-ab66-4e96-8e91-ad608e07a6f6-kube-api-access-7plnp" (OuterVolumeSpecName: "kube-api-access-7plnp") pod "60773ed6-ab66-4e96-8e91-ad608e07a6f6" (UID: "60773ed6-ab66-4e96-8e91-ad608e07a6f6"). InnerVolumeSpecName "kube-api-access-7plnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:18:40.742551 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.742516    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-trusted-ca-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:18:40.742551 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.742544    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-oauth-config\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:18:40.742551 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.742554    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-serving-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:18:40.742551 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.742562    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-oauth-serving-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:18:40.742809 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.742572    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:18:40.742809 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.742581    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60773ed6-ab66-4e96-8e91-ad608e07a6f6-console-config\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:18:40.742809 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:40.742588    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7plnp\" (UniqueName: \"kubernetes.io/projected/60773ed6-ab66-4e96-8e91-ad608e07a6f6-kube-api-access-7plnp\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:18:41.197891 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.197856    2572 generic.go:356] "Generic (PLEG): container finished" podID="60773ed6-ab66-4e96-8e91-ad608e07a6f6" containerID="b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7" exitCode=0
Jul 16 21:18:41.198069 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.197905    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d69fc5976-xlds9"
Jul 16 21:18:41.198069 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.197921    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d69fc5976-xlds9" event={"ID":"60773ed6-ab66-4e96-8e91-ad608e07a6f6","Type":"ContainerDied","Data":"b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7"}
Jul 16 21:18:41.198069 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.197963    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d69fc5976-xlds9" event={"ID":"60773ed6-ab66-4e96-8e91-ad608e07a6f6","Type":"ContainerDied","Data":"d8986ac9c4423374c146875052ab04342e933576ccdd8f01a308b7f998923f8c"}
Jul 16 21:18:41.198069 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.197982    2572 scope.go:117] "RemoveContainer" containerID="b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7"
Jul 16 21:18:41.206987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.206968    2572 scope.go:117] "RemoveContainer" containerID="b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7"
Jul 16 21:18:41.207274 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:18:41.207254    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7\": container with ID starting with b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7 not found: ID does not exist" containerID="b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7"
Jul 16 21:18:41.207315 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.207282    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7"} err="failed to get container status \"b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7\": rpc error: code = NotFound desc = could not find container \"b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7\": container with ID starting with b6dc853c1bd19f3d0aa3d5d349a5a697404c5991e99c23c27cfe374c4a82f7b7 not found: ID does not exist"
Jul 16 21:18:41.223508 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.223475    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d69fc5976-xlds9"]
Jul 16 21:18:41.229644 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.229607    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d69fc5976-xlds9"]
Jul 16 21:18:41.247986 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:18:41.247956    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60773ed6-ab66-4e96-8e91-ad608e07a6f6" path="/var/lib/kubelet/pods/60773ed6-ab66-4e96-8e91-ad608e07a6f6/volumes"
Jul 16 21:19:51.796140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.796098    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"]
Jul 16 21:19:51.796658 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.796410    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60773ed6-ab66-4e96-8e91-ad608e07a6f6" containerName="console"
Jul 16 21:19:51.796658 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.796422    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="60773ed6-ab66-4e96-8e91-ad608e07a6f6" containerName="console"
Jul 16 21:19:51.796658 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.796470    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="60773ed6-ab66-4e96-8e91-ad608e07a6f6" containerName="console"
Jul 16 21:19:51.800663 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.800645    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:51.803049 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.803018    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 21:19:51.803883 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.803862    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 21:19:51.803962 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.803862    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vkxpt\""
Jul 16 21:19:51.808755 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.808727    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"]
Jul 16 21:19:51.912511 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.912475    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:51.912694 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.912542    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:51.912694 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:51.912596    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbq8k\" (UniqueName: \"kubernetes.io/projected/bb7dbb81-f313-4efc-9053-639bf17b8a2a-kube-api-access-cbq8k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:52.013949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.013901    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:52.013949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.013954    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbq8k\" (UniqueName: \"kubernetes.io/projected/bb7dbb81-f313-4efc-9053-639bf17b8a2a-kube-api-access-cbq8k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:52.014154 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.013988    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:52.014312 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.014294    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:52.014422 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.014341    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:52.022817 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.022791    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbq8k\" (UniqueName: \"kubernetes.io/projected/bb7dbb81-f313-4efc-9053-639bf17b8a2a-kube-api-access-cbq8k\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:52.110061 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.109975    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:19:52.242091 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.242059    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"]
Jul 16 21:19:52.245948 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:19:52.245920    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb7dbb81_f313_4efc_9053_639bf17b8a2a.slice/crio-53054ee87a63ac3c8dd0ffe3256b8477450fccbd1e5addd135395022b1943729 WatchSource:0}: Error finding container 53054ee87a63ac3c8dd0ffe3256b8477450fccbd1e5addd135395022b1943729: Status 404 returned error can't find the container with id 53054ee87a63ac3c8dd0ffe3256b8477450fccbd1e5addd135395022b1943729
Jul 16 21:19:52.461857 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:52.461763    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4" event={"ID":"bb7dbb81-f313-4efc-9053-639bf17b8a2a","Type":"ContainerStarted","Data":"53054ee87a63ac3c8dd0ffe3256b8477450fccbd1e5addd135395022b1943729"}
Jul 16 21:19:59.490419 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:59.490378    2572 generic.go:356] "Generic (PLEG): container finished" podID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerID="378ac4c3515975cfb96de426361bfdbfc739d087c18736977ab8d937df15c3bd" exitCode=0
Jul 16 21:19:59.490927 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:19:59.490458    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4" event={"ID":"bb7dbb81-f313-4efc-9053-639bf17b8a2a","Type":"ContainerDied","Data":"378ac4c3515975cfb96de426361bfdbfc739d087c18736977ab8d937df15c3bd"}
Jul 16 21:20:01.500992 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:01.500966    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4" event={"ID":"bb7dbb81-f313-4efc-9053-639bf17b8a2a","Type":"ContainerStarted","Data":"0c71d1ae8ea04b68d2f1657a0efc39e14e2ad38f6cc89c73e332becb7efa0b9b"}
Jul 16 21:20:02.506499 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:02.506463    2572 generic.go:356] "Generic (PLEG): container finished" podID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerID="0c71d1ae8ea04b68d2f1657a0efc39e14e2ad38f6cc89c73e332becb7efa0b9b" exitCode=0
Jul 16 21:20:02.506881 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:02.506537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4" event={"ID":"bb7dbb81-f313-4efc-9053-639bf17b8a2a","Type":"ContainerDied","Data":"0c71d1ae8ea04b68d2f1657a0efc39e14e2ad38f6cc89c73e332becb7efa0b9b"}
Jul 16 21:20:04.276974 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.276934    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65f7b757f5-mm2tr"]
Jul 16 21:20:04.279440 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.279414    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.291594 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.291564    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65f7b757f5-mm2tr"]
Jul 16 21:20:04.422042 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.422005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-serving-cert\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.422203 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.422066    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-oauth-serving-cert\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.422203 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.422126    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvt2l\" (UniqueName: \"kubernetes.io/projected/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-kube-api-access-vvt2l\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.422203 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.422190    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-oauth-config\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.422386 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.422207    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-service-ca\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.422386 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.422229    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-config\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.422386 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.422273    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-trusted-ca-bundle\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.523398 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.523359    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-oauth-serving-cert\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.523573 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.523405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvt2l\" (UniqueName: \"kubernetes.io/projected/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-kube-api-access-vvt2l\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.523573 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.523458    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-oauth-config\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.523573 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.523482    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-service-ca\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.523735 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.523629    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-config\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.523735 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.523688    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-trusted-ca-bundle\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.523838 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.523738    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-serving-cert\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.524258 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.524196    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-oauth-serving-cert\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.524258 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.524225    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-service-ca\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.524467 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.524310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-config\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.524675 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.524651    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-trusted-ca-bundle\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.526192 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.526162    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-oauth-config\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.526410 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.526394    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-serving-cert\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.533234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.533171    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvt2l\" (UniqueName: \"kubernetes.io/projected/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-kube-api-access-vvt2l\") pod \"console-65f7b757f5-mm2tr\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") " pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.592793 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.592755    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:04.738834 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:04.738808    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65f7b757f5-mm2tr"]
Jul 16 21:20:04.741468 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:20:04.741411    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24eaf0a_81cc_48d3_8e00_e1b7a9617868.slice/crio-e54915b26fe8d083041ecaae8bb845593dba2d0129ccf28cce7edc9f5429c40d WatchSource:0}: Error finding container e54915b26fe8d083041ecaae8bb845593dba2d0129ccf28cce7edc9f5429c40d: Status 404 returned error can't find the container with id e54915b26fe8d083041ecaae8bb845593dba2d0129ccf28cce7edc9f5429c40d
Jul 16 21:20:05.523133 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:05.523093    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f7b757f5-mm2tr" event={"ID":"e24eaf0a-81cc-48d3-8e00-e1b7a9617868","Type":"ContainerStarted","Data":"88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7"}
Jul 16 21:20:05.523633 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:05.523157    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f7b757f5-mm2tr" event={"ID":"e24eaf0a-81cc-48d3-8e00-e1b7a9617868","Type":"ContainerStarted","Data":"e54915b26fe8d083041ecaae8bb845593dba2d0129ccf28cce7edc9f5429c40d"}
Jul 16 21:20:05.543260 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:05.543199    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65f7b757f5-mm2tr" podStartSLOduration=1.543177988 podStartE2EDuration="1.543177988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:20:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:05.54202262 +0000 UTC m=+271.053182356" watchObservedRunningTime="2026-07-16 21:20:05.543177988 +0000 UTC m=+271.054337724"
Jul 16 21:20:10.546197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:10.546162    2572 generic.go:356] "Generic (PLEG): container finished" podID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerID="f0480a809d9a5a5a6039135873ef6acd882d7f4d5969084d4fefa3c0626a5e22" exitCode=0
Jul 16 21:20:10.546637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:10.546255    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4" event={"ID":"bb7dbb81-f313-4efc-9053-639bf17b8a2a","Type":"ContainerDied","Data":"f0480a809d9a5a5a6039135873ef6acd882d7f4d5969084d4fefa3c0626a5e22"}
Jul 16 21:20:11.685830 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.685805    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:20:11.785676 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.785642    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-bundle\") pod \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") "
Jul 16 21:20:11.785676 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.785683    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbq8k\" (UniqueName: \"kubernetes.io/projected/bb7dbb81-f313-4efc-9053-639bf17b8a2a-kube-api-access-cbq8k\") pod \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") "
Jul 16 21:20:11.785902 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.785748    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-util\") pod \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\" (UID: \"bb7dbb81-f313-4efc-9053-639bf17b8a2a\") "
Jul 16 21:20:11.786372 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.786311    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-bundle" (OuterVolumeSpecName: "bundle") pod "bb7dbb81-f313-4efc-9053-639bf17b8a2a" (UID: "bb7dbb81-f313-4efc-9053-639bf17b8a2a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:11.787878 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.787855    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb7dbb81-f313-4efc-9053-639bf17b8a2a-kube-api-access-cbq8k" (OuterVolumeSpecName: "kube-api-access-cbq8k") pod "bb7dbb81-f313-4efc-9053-639bf17b8a2a" (UID: "bb7dbb81-f313-4efc-9053-639bf17b8a2a"). InnerVolumeSpecName "kube-api-access-cbq8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:20:11.792729 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.792698    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-util" (OuterVolumeSpecName: "util") pod "bb7dbb81-f313-4efc-9053-639bf17b8a2a" (UID: "bb7dbb81-f313-4efc-9053-639bf17b8a2a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:11.887194 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.887106    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-util\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:11.887194 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.887136    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb7dbb81-f313-4efc-9053-639bf17b8a2a-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:11.887194 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:11.887149    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbq8k\" (UniqueName: \"kubernetes.io/projected/bb7dbb81-f313-4efc-9053-639bf17b8a2a-kube-api-access-cbq8k\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:12.555626 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:12.555591    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4" event={"ID":"bb7dbb81-f313-4efc-9053-639bf17b8a2a","Type":"ContainerDied","Data":"53054ee87a63ac3c8dd0ffe3256b8477450fccbd1e5addd135395022b1943729"}
Jul 16 21:20:12.555626 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:12.555615    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hl8t4"
Jul 16 21:20:12.555827 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:12.555642    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="53054ee87a63ac3c8dd0ffe3256b8477450fccbd1e5addd135395022b1943729"
Jul 16 21:20:14.593918 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:14.593874    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:14.594292 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:14.593945    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:14.598666 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:14.598639    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:15.570488 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:15.570463    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:20:15.623551 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:15.623522    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6768c779bf-jdl42"]
Jul 16 21:20:19.751158 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.751080    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"]
Jul 16 21:20:19.751525 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.751390    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerName="util"
Jul 16 21:20:19.751525 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.751405    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerName="util"
Jul 16 21:20:19.751525 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.751416    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerName="extract"
Jul 16 21:20:19.751525 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.751423    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerName="extract"
Jul 16 21:20:19.751525 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.751442    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerName="pull"
Jul 16 21:20:19.751525 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.751448    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerName="pull"
Jul 16 21:20:19.751525 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.751499    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb7dbb81-f313-4efc-9053-639bf17b8a2a" containerName="extract"
Jul 16 21:20:19.754593 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.754574    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"
Jul 16 21:20:19.757680 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.757656    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 16 21:20:19.758160 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.758143    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 16 21:20:19.758220 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.758159    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4kzmm\""
Jul 16 21:20:19.768697 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.768670    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"]
Jul 16 21:20:19.853808 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.853769    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f563294e-5847-4dc1-ab90-8775b4f87009-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hfwp2\" (UID: \"f563294e-5847-4dc1-ab90-8775b4f87009\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"
Jul 16 21:20:19.853986 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.853832    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n7pl\" (UniqueName: \"kubernetes.io/projected/f563294e-5847-4dc1-ab90-8775b4f87009-kube-api-access-8n7pl\") pod \"cert-manager-operator-controller-manager-55c444bf86-hfwp2\" (UID: \"f563294e-5847-4dc1-ab90-8775b4f87009\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"
Jul 16 21:20:19.955118 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.955078    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n7pl\" (UniqueName: \"kubernetes.io/projected/f563294e-5847-4dc1-ab90-8775b4f87009-kube-api-access-8n7pl\") pod \"cert-manager-operator-controller-manager-55c444bf86-hfwp2\" (UID: \"f563294e-5847-4dc1-ab90-8775b4f87009\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"
Jul 16 21:20:19.955233 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.955140    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f563294e-5847-4dc1-ab90-8775b4f87009-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hfwp2\" (UID: \"f563294e-5847-4dc1-ab90-8775b4f87009\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"
Jul 16 21:20:19.955533 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.955518    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f563294e-5847-4dc1-ab90-8775b4f87009-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hfwp2\" (UID: \"f563294e-5847-4dc1-ab90-8775b4f87009\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"
Jul 16 21:20:19.964108 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:19.964077    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n7pl\" (UniqueName: \"kubernetes.io/projected/f563294e-5847-4dc1-ab90-8775b4f87009-kube-api-access-8n7pl\") pod \"cert-manager-operator-controller-manager-55c444bf86-hfwp2\" (UID: \"f563294e-5847-4dc1-ab90-8775b4f87009\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"
Jul 16 21:20:20.066211 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:20.066172    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"
Jul 16 21:20:20.204846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:20.204720    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2"]
Jul 16 21:20:20.207185 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:20:20.207154    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf563294e_5847_4dc1_ab90_8775b4f87009.slice/crio-abd0c5523afbb02845911d558192d20d98ccc7b2fd28a7d11585070fd92801f7 WatchSource:0}: Error finding container abd0c5523afbb02845911d558192d20d98ccc7b2fd28a7d11585070fd92801f7: Status 404 returned error can't find the container with id abd0c5523afbb02845911d558192d20d98ccc7b2fd28a7d11585070fd92801f7
Jul 16 21:20:20.586915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:20.586877    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2" event={"ID":"f563294e-5847-4dc1-ab90-8775b4f87009","Type":"ContainerStarted","Data":"abd0c5523afbb02845911d558192d20d98ccc7b2fd28a7d11585070fd92801f7"}
Jul 16 21:20:22.597394 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:22.597346    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2" event={"ID":"f563294e-5847-4dc1-ab90-8775b4f87009","Type":"ContainerStarted","Data":"f8a4c8b331ee3661dad8c48795fcaac50397923dd5ea708f5537b27617ac8001"}
Jul 16 21:20:22.620670 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:22.620612    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hfwp2" podStartSLOduration=1.418420493 podStartE2EDuration="3.620591374s" totalImagesPullingTime="2.202170881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:22.618694889 +0000 UTC m=+288.129854622" watchObservedRunningTime="2026-07-16 21:20:22.620591374 +0000 UTC m=+288.131751108"
Jul 16 21:20:23.526201 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.526164    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mv4fl"]
Jul 16 21:20:23.529518 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.529492    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:23.533106 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.532899    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2vfgm\""
Jul 16 21:20:23.533106 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.532961    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 16 21:20:23.533106 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.533011    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 16 21:20:23.538071 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.538043    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mv4fl"]
Jul 16 21:20:23.582710 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.582679    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27b96b4e-8388-4ab6-a395-70705e7430ac-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mv4fl\" (UID: \"27b96b4e-8388-4ab6-a395-70705e7430ac\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:23.582878 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.582723    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh7j8\" (UniqueName: \"kubernetes.io/projected/27b96b4e-8388-4ab6-a395-70705e7430ac-kube-api-access-sh7j8\") pod \"cert-manager-webhook-6486ffd997-mv4fl\" (UID: \"27b96b4e-8388-4ab6-a395-70705e7430ac\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:23.684143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.684112    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27b96b4e-8388-4ab6-a395-70705e7430ac-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mv4fl\" (UID: \"27b96b4e-8388-4ab6-a395-70705e7430ac\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:23.684571 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.684158    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh7j8\" (UniqueName: \"kubernetes.io/projected/27b96b4e-8388-4ab6-a395-70705e7430ac-kube-api-access-sh7j8\") pod \"cert-manager-webhook-6486ffd997-mv4fl\" (UID: \"27b96b4e-8388-4ab6-a395-70705e7430ac\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:23.695384 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.695330    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27b96b4e-8388-4ab6-a395-70705e7430ac-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mv4fl\" (UID: \"27b96b4e-8388-4ab6-a395-70705e7430ac\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:23.695384 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.695369    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh7j8\" (UniqueName: \"kubernetes.io/projected/27b96b4e-8388-4ab6-a395-70705e7430ac-kube-api-access-sh7j8\") pod \"cert-manager-webhook-6486ffd997-mv4fl\" (UID: \"27b96b4e-8388-4ab6-a395-70705e7430ac\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:23.712032 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.711997    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"]
Jul 16 21:20:23.715382 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.715340    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.717636 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.717619    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 21:20:23.718395 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.718380    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 21:20:23.718495 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.718459    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vkxpt\""
Jul 16 21:20:23.735013 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.734976    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"]
Jul 16 21:20:23.785256 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.785170    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jvqn\" (UniqueName: \"kubernetes.io/projected/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-kube-api-access-8jvqn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.785256 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.785244    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.785465 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.785267    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.850238 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.850201    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:23.886589 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.886546    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jvqn\" (UniqueName: \"kubernetes.io/projected/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-kube-api-access-8jvqn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.886788 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.886634    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.886788 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.886660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.887029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.887009    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.887093 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.887027    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.895752 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.895725    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jvqn\" (UniqueName: \"kubernetes.io/projected/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-kube-api-access-8jvqn\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:23.983925 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:23.983891    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mv4fl"]
Jul 16 21:20:23.987096 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:20:23.987069    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27b96b4e_8388_4ab6_a395_70705e7430ac.slice/crio-5e8f31923b8e00c2cf3ee56f7153bd0cb400be4b6c0ae1a01e5ea8cd26c92ba3 WatchSource:0}: Error finding container 5e8f31923b8e00c2cf3ee56f7153bd0cb400be4b6c0ae1a01e5ea8cd26c92ba3: Status 404 returned error can't find the container with id 5e8f31923b8e00c2cf3ee56f7153bd0cb400be4b6c0ae1a01e5ea8cd26c92ba3
Jul 16 21:20:24.025257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:24.025222    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:24.162925 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:24.162894    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"]
Jul 16 21:20:24.166686 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:20:24.166655    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e242f0e_d5b6_4234_a3fa_1dedfd27faa0.slice/crio-1eec56c24c732364cad51f35f1449a8c068dc48c4c2994e7d9a5b4fef9a91790 WatchSource:0}: Error finding container 1eec56c24c732364cad51f35f1449a8c068dc48c4c2994e7d9a5b4fef9a91790: Status 404 returned error can't find the container with id 1eec56c24c732364cad51f35f1449a8c068dc48c4c2994e7d9a5b4fef9a91790
Jul 16 21:20:24.606815 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:24.606776    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl" event={"ID":"27b96b4e-8388-4ab6-a395-70705e7430ac","Type":"ContainerStarted","Data":"5e8f31923b8e00c2cf3ee56f7153bd0cb400be4b6c0ae1a01e5ea8cd26c92ba3"}
Jul 16 21:20:24.608218 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:24.608187    2572 generic.go:356] "Generic (PLEG): container finished" podID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerID="209e63754b6a7a1b1193fe2ff5ab0aa00ff46c3cdd4caabf47d5a84a76e2a9df" exitCode=0
Jul 16 21:20:24.608386 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:24.608226    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd" event={"ID":"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0","Type":"ContainerDied","Data":"209e63754b6a7a1b1193fe2ff5ab0aa00ff46c3cdd4caabf47d5a84a76e2a9df"}
Jul 16 21:20:24.608386 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:24.608297    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd" event={"ID":"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0","Type":"ContainerStarted","Data":"1eec56c24c732364cad51f35f1449a8c068dc48c4c2994e7d9a5b4fef9a91790"}
Jul 16 21:20:26.110320 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.110280    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"]
Jul 16 21:20:26.113414 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.113389    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"
Jul 16 21:20:26.115748 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.115722    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hb2hq\""
Jul 16 21:20:26.127852 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.127820    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"]
Jul 16 21:20:26.205926 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.205887    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff668a5c-e7da-4b10-af26-d854a0b3820c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xbxn4\" (UID: \"ff668a5c-e7da-4b10-af26-d854a0b3820c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"
Jul 16 21:20:26.206116 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.205944    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82hbq\" (UniqueName: \"kubernetes.io/projected/ff668a5c-e7da-4b10-af26-d854a0b3820c-kube-api-access-82hbq\") pod \"cert-manager-cainjector-559cdd8c87-xbxn4\" (UID: \"ff668a5c-e7da-4b10-af26-d854a0b3820c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"
Jul 16 21:20:26.307341 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.307297    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff668a5c-e7da-4b10-af26-d854a0b3820c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xbxn4\" (UID: \"ff668a5c-e7da-4b10-af26-d854a0b3820c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"
Jul 16 21:20:26.307543 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.307394    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82hbq\" (UniqueName: \"kubernetes.io/projected/ff668a5c-e7da-4b10-af26-d854a0b3820c-kube-api-access-82hbq\") pod \"cert-manager-cainjector-559cdd8c87-xbxn4\" (UID: \"ff668a5c-e7da-4b10-af26-d854a0b3820c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"
Jul 16 21:20:26.316202 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.316171    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff668a5c-e7da-4b10-af26-d854a0b3820c-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xbxn4\" (UID: \"ff668a5c-e7da-4b10-af26-d854a0b3820c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"
Jul 16 21:20:26.316401 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.316377    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82hbq\" (UniqueName: \"kubernetes.io/projected/ff668a5c-e7da-4b10-af26-d854a0b3820c-kube-api-access-82hbq\") pod \"cert-manager-cainjector-559cdd8c87-xbxn4\" (UID: \"ff668a5c-e7da-4b10-af26-d854a0b3820c\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"
Jul 16 21:20:26.424239 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:26.424150    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"
Jul 16 21:20:27.258230 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:27.258167    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4"]
Jul 16 21:20:27.259773 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:20:27.259745    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff668a5c_e7da_4b10_af26_d854a0b3820c.slice/crio-d4e2f8ae6f8f09c729b715dac7d9cfd2de6104ba2b7d1720d0b6eed76a22c882 WatchSource:0}: Error finding container d4e2f8ae6f8f09c729b715dac7d9cfd2de6104ba2b7d1720d0b6eed76a22c882: Status 404 returned error can't find the container with id d4e2f8ae6f8f09c729b715dac7d9cfd2de6104ba2b7d1720d0b6eed76a22c882
Jul 16 21:20:27.624915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:27.624875    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl" event={"ID":"27b96b4e-8388-4ab6-a395-70705e7430ac","Type":"ContainerStarted","Data":"964281d92acdd093c5ac9ecab24174fd45b94390a25cdc2091e9d0ec29541402"}
Jul 16 21:20:27.625101 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:27.624959    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:27.626877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:27.626845    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4" event={"ID":"ff668a5c-e7da-4b10-af26-d854a0b3820c","Type":"ContainerStarted","Data":"e0036fc807150fb1c685a4b4bb03593b96e22456edbd93144acb3a118bd493d1"}
Jul 16 21:20:27.627002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:27.626888    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4" event={"ID":"ff668a5c-e7da-4b10-af26-d854a0b3820c","Type":"ContainerStarted","Data":"d4e2f8ae6f8f09c729b715dac7d9cfd2de6104ba2b7d1720d0b6eed76a22c882"}
Jul 16 21:20:27.644133 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:27.644027    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl" podStartSLOduration=1.450734501 podStartE2EDuration="4.64400909s" totalImagesPullingTime="3.193274589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:27.643212156 +0000 UTC m=+293.154371891" watchObservedRunningTime="2026-07-16 21:20:27.64400909 +0000 UTC m=+293.155168824"
Jul 16 21:20:27.668176 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:27.668124    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xbxn4" podStartSLOduration=1.668103539 podStartE2EDuration="1.668103539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:20:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:27.6664652 +0000 UTC m=+293.177624942" watchObservedRunningTime="2026-07-16 21:20:27.668103539 +0000 UTC m=+293.179263273"
Jul 16 21:20:29.637575 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:29.637540    2572 generic.go:356] "Generic (PLEG): container finished" podID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerID="04f20983d88e2dae8ed858ed8a7583c23ee04c2eebd5b7c4949e22179f240373" exitCode=0
Jul 16 21:20:29.637961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:29.637589    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd" event={"ID":"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0","Type":"ContainerDied","Data":"04f20983d88e2dae8ed858ed8a7583c23ee04c2eebd5b7c4949e22179f240373"}
Jul 16 21:20:30.644447 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:30.644409    2572 generic.go:356] "Generic (PLEG): container finished" podID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerID="f77fd9794323ab5fb2aafdbfa868dc03c635fddca4c7b85b1794c149ebe80e31" exitCode=0
Jul 16 21:20:30.644825 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:30.644458    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd" event={"ID":"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0","Type":"ContainerDied","Data":"f77fd9794323ab5fb2aafdbfa868dc03c635fddca4c7b85b1794c149ebe80e31"}
Jul 16 21:20:31.806644 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.806606    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:31.855267 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.855236    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-util\") pod \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") "
Jul 16 21:20:31.855436 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.855339    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jvqn\" (UniqueName: \"kubernetes.io/projected/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-kube-api-access-8jvqn\") pod \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") "
Jul 16 21:20:31.855436 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.855392    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-bundle\") pod \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\" (UID: \"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0\") "
Jul 16 21:20:31.855805 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.855775    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-bundle" (OuterVolumeSpecName: "bundle") pod "1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" (UID: "1e242f0e-d5b6-4234-a3fa-1dedfd27faa0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:31.857635 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.857610    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-kube-api-access-8jvqn" (OuterVolumeSpecName: "kube-api-access-8jvqn") pod "1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" (UID: "1e242f0e-d5b6-4234-a3fa-1dedfd27faa0"). InnerVolumeSpecName "kube-api-access-8jvqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:20:31.859960 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.859939    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-util" (OuterVolumeSpecName: "util") pod "1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" (UID: "1e242f0e-d5b6-4234-a3fa-1dedfd27faa0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:31.956811 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.956728    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jvqn\" (UniqueName: \"kubernetes.io/projected/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-kube-api-access-8jvqn\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:31.956811 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.956761    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:31.956811 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:31.956770    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e242f0e-d5b6-4234-a3fa-1dedfd27faa0-util\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:32.653939 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:32.653907    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd" event={"ID":"1e242f0e-d5b6-4234-a3fa-1dedfd27faa0","Type":"ContainerDied","Data":"1eec56c24c732364cad51f35f1449a8c068dc48c4c2994e7d9a5b4fef9a91790"}
Jul 16 21:20:32.654106 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:32.653949    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2gsgd"
Jul 16 21:20:32.654106 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:32.653952    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1eec56c24c732364cad51f35f1449a8c068dc48c4c2994e7d9a5b4fef9a91790"
Jul 16 21:20:33.635641 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:33.635614    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mv4fl"
Jul 16 21:20:34.800748 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.800712    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hr845"]
Jul 16 21:20:34.801208 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.801178    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerName="pull"
Jul 16 21:20:34.801208 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.801196    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerName="pull"
Jul 16 21:20:34.801323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.801217    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerName="extract"
Jul 16 21:20:34.801323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.801225    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerName="extract"
Jul 16 21:20:34.801323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.801244    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerName="util"
Jul 16 21:20:34.801323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.801253    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerName="util"
Jul 16 21:20:34.801540 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.801338    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e242f0e-d5b6-4234-a3fa-1dedfd27faa0" containerName="extract"
Jul 16 21:20:34.804337 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.804310    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hr845"
Jul 16 21:20:34.806614 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.806588    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mw8s6\""
Jul 16 21:20:34.814074 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.814042    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hr845"]
Jul 16 21:20:34.880892 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.880851    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/979f9dcf-e53a-4ac9-ba45-a2fe66faad63-bound-sa-token\") pod \"cert-manager-55b4d954d5-hr845\" (UID: \"979f9dcf-e53a-4ac9-ba45-a2fe66faad63\") " pod="cert-manager/cert-manager-55b4d954d5-hr845"
Jul 16 21:20:34.881069 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.880923    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwl6z\" (UniqueName: \"kubernetes.io/projected/979f9dcf-e53a-4ac9-ba45-a2fe66faad63-kube-api-access-bwl6z\") pod \"cert-manager-55b4d954d5-hr845\" (UID: \"979f9dcf-e53a-4ac9-ba45-a2fe66faad63\") " pod="cert-manager/cert-manager-55b4d954d5-hr845"
Jul 16 21:20:34.982227 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.982189    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwl6z\" (UniqueName: \"kubernetes.io/projected/979f9dcf-e53a-4ac9-ba45-a2fe66faad63-kube-api-access-bwl6z\") pod \"cert-manager-55b4d954d5-hr845\" (UID: \"979f9dcf-e53a-4ac9-ba45-a2fe66faad63\") " pod="cert-manager/cert-manager-55b4d954d5-hr845"
Jul 16 21:20:34.982418 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.982249    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/979f9dcf-e53a-4ac9-ba45-a2fe66faad63-bound-sa-token\") pod \"cert-manager-55b4d954d5-hr845\" (UID: \"979f9dcf-e53a-4ac9-ba45-a2fe66faad63\") " pod="cert-manager/cert-manager-55b4d954d5-hr845"
Jul 16 21:20:34.993102 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.993066    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/979f9dcf-e53a-4ac9-ba45-a2fe66faad63-bound-sa-token\") pod \"cert-manager-55b4d954d5-hr845\" (UID: \"979f9dcf-e53a-4ac9-ba45-a2fe66faad63\") " pod="cert-manager/cert-manager-55b4d954d5-hr845"
Jul 16 21:20:34.993230 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:34.993138    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwl6z\" (UniqueName: \"kubernetes.io/projected/979f9dcf-e53a-4ac9-ba45-a2fe66faad63-kube-api-access-bwl6z\") pod \"cert-manager-55b4d954d5-hr845\" (UID: \"979f9dcf-e53a-4ac9-ba45-a2fe66faad63\") " pod="cert-manager/cert-manager-55b4d954d5-hr845"
Jul 16 21:20:35.093559 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:35.093485    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 21:20:35.117232 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:35.117204    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mw8s6\""
Jul 16 21:20:35.124405 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:35.124376    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hr845"
Jul 16 21:20:35.265916 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:35.258446    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hr845"]
Jul 16 21:20:35.265916 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:20:35.261400    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod979f9dcf_e53a_4ac9_ba45_a2fe66faad63.slice/crio-b6a91aee33aa0c364baffde19119dc0a879c473c882666a064d2d82cc857b9dd WatchSource:0}: Error finding container b6a91aee33aa0c364baffde19119dc0a879c473c882666a064d2d82cc857b9dd: Status 404 returned error can't find the container with id b6a91aee33aa0c364baffde19119dc0a879c473c882666a064d2d82cc857b9dd
Jul 16 21:20:35.668213 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:35.668182    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hr845" event={"ID":"979f9dcf-e53a-4ac9-ba45-a2fe66faad63","Type":"ContainerStarted","Data":"1f45d3fae900ff2ca354dace6331df935f0620a38997082afba6a1d24ebfe72b"}
Jul 16 21:20:35.668213 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:35.668219    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hr845" event={"ID":"979f9dcf-e53a-4ac9-ba45-a2fe66faad63","Type":"ContainerStarted","Data":"b6a91aee33aa0c364baffde19119dc0a879c473c882666a064d2d82cc857b9dd"}
Jul 16 21:20:35.689764 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:35.689714    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hr845" podStartSLOduration=1.689699056 podStartE2EDuration="1.689699056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:20:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:35.688482533 +0000 UTC m=+301.199642270" watchObservedRunningTime="2026-07-16 21:20:35.689699056 +0000 UTC m=+301.200858791"
Jul 16 21:20:40.648595 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:40.648537    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6768c779bf-jdl42" podUID="6380c25c-646c-4eb8-9c06-d75a3a3e43d0" containerName="console" containerID="cri-o://f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157" gracePeriod=15
Jul 16 21:20:40.902214 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:40.902147    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:20:41.030859 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.030815    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hscph\" (UniqueName: \"kubernetes.io/projected/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-kube-api-access-hscph\") pod \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") "
Jul 16 21:20:41.030859 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.030871    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-service-ca\") pod \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") "
Jul 16 21:20:41.031073 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.030892    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-config\") pod \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") "
Jul 16 21:20:41.031073 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.030948    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-serving-cert\") pod \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") "
Jul 16 21:20:41.031073 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.030974    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-oauth-config\") pod \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") "
Jul 16 21:20:41.031073 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.030997    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-oauth-serving-cert\") pod \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") "
Jul 16 21:20:41.031073 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.031023    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-trusted-ca-bundle\") pod \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\" (UID: \"6380c25c-646c-4eb8-9c06-d75a3a3e43d0\") "
Jul 16 21:20:41.031533 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.031481    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-config" (OuterVolumeSpecName: "console-config") pod "6380c25c-646c-4eb8-9c06-d75a3a3e43d0" (UID: "6380c25c-646c-4eb8-9c06-d75a3a3e43d0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:20:41.031533 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.031509    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6380c25c-646c-4eb8-9c06-d75a3a3e43d0" (UID: "6380c25c-646c-4eb8-9c06-d75a3a3e43d0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:20:41.031710 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.031538    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6380c25c-646c-4eb8-9c06-d75a3a3e43d0" (UID: "6380c25c-646c-4eb8-9c06-d75a3a3e43d0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:20:41.031710 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.031590    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-service-ca" (OuterVolumeSpecName: "service-ca") pod "6380c25c-646c-4eb8-9c06-d75a3a3e43d0" (UID: "6380c25c-646c-4eb8-9c06-d75a3a3e43d0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:20:41.033084 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.033064    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6380c25c-646c-4eb8-9c06-d75a3a3e43d0" (UID: "6380c25c-646c-4eb8-9c06-d75a3a3e43d0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:20:41.033168 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.033064    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6380c25c-646c-4eb8-9c06-d75a3a3e43d0" (UID: "6380c25c-646c-4eb8-9c06-d75a3a3e43d0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:20:41.033168 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.033145    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-kube-api-access-hscph" (OuterVolumeSpecName: "kube-api-access-hscph") pod "6380c25c-646c-4eb8-9c06-d75a3a3e43d0" (UID: "6380c25c-646c-4eb8-9c06-d75a3a3e43d0"). InnerVolumeSpecName "kube-api-access-hscph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:20:41.132278 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.132234    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:41.132278 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.132280    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-config\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:41.132490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.132294    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-serving-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:41.132490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.132308    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-console-oauth-config\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:41.132490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.132322    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-oauth-serving-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:41.132490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.132333    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-trusted-ca-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:41.132490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.132346    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hscph\" (UniqueName: \"kubernetes.io/projected/6380c25c-646c-4eb8-9c06-d75a3a3e43d0-kube-api-access-hscph\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:41.694284 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.694244    2572 generic.go:356] "Generic (PLEG): container finished" podID="6380c25c-646c-4eb8-9c06-d75a3a3e43d0" containerID="f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157" exitCode=0
Jul 16 21:20:41.694765 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.694310    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6768c779bf-jdl42"
Jul 16 21:20:41.694765 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.694330    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6768c779bf-jdl42" event={"ID":"6380c25c-646c-4eb8-9c06-d75a3a3e43d0","Type":"ContainerDied","Data":"f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157"}
Jul 16 21:20:41.694765 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.694404    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6768c779bf-jdl42" event={"ID":"6380c25c-646c-4eb8-9c06-d75a3a3e43d0","Type":"ContainerDied","Data":"866d8c8b1de789c5e8ff3d674b63eb786db4047d1826002ce3139c4222ea3508"}
Jul 16 21:20:41.694765 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.694425    2572 scope.go:117] "RemoveContainer" containerID="f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157"
Jul 16 21:20:41.702572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.702553    2572 scope.go:117] "RemoveContainer" containerID="f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157"
Jul 16 21:20:41.702849 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:20:41.702830    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157\": container with ID starting with f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157 not found: ID does not exist" containerID="f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157"
Jul 16 21:20:41.702891 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.702860    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157"} err="failed to get container status \"f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157\": rpc error: code = NotFound desc = could not find container \"f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157\": container with ID starting with f64abbd622a1407b6e1ba65570a981b4227b97b0a9b506bd2af2810f32383157 not found: ID does not exist"
Jul 16 21:20:41.727713 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.727676    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6768c779bf-jdl42"]
Jul 16 21:20:41.735488 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:41.735461    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6768c779bf-jdl42"]
Jul 16 21:20:43.186318 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.186288    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"]
Jul 16 21:20:43.186701 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.186642    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6380c25c-646c-4eb8-9c06-d75a3a3e43d0" containerName="console"
Jul 16 21:20:43.186701 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.186654    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6380c25c-646c-4eb8-9c06-d75a3a3e43d0" containerName="console"
Jul 16 21:20:43.186773 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.186719    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6380c25c-646c-4eb8-9c06-d75a3a3e43d0" containerName="console"
Jul 16 21:20:43.222160 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.222117    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"]
Jul 16 21:20:43.222160 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.222163    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.224791 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.224772    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 16 21:20:43.224911 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.224889    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vkxpt\""
Jul 16 21:20:43.224961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.224927    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 16 21:20:43.249394 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.249345    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6380c25c-646c-4eb8-9c06-d75a3a3e43d0" path="/var/lib/kubelet/pods/6380c25c-646c-4eb8-9c06-d75a3a3e43d0/volumes"
Jul 16 21:20:43.350188 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.350152    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sv9h\" (UniqueName: \"kubernetes.io/projected/ee959131-408e-4cde-8a52-1599d47e9173-kube-api-access-5sv9h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.350399 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.350233    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.350399 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.350271    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.451419 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.451314    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sv9h\" (UniqueName: \"kubernetes.io/projected/ee959131-408e-4cde-8a52-1599d47e9173-kube-api-access-5sv9h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.451590 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.451476    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.451590 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.451515    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.451913 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.451891    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.451955 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.451941    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.460414 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.460386    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sv9h\" (UniqueName: \"kubernetes.io/projected/ee959131-408e-4cde-8a52-1599d47e9173-kube-api-access-5sv9h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.531791 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.531761    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:43.679041 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.679010    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"]
Jul 16 21:20:43.682398 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:20:43.682334    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee959131_408e_4cde_8a52_1599d47e9173.slice/crio-be443430a96598d207e76e5b794f8f7aa5a7ea89c93b16a5061dfe673432c076 WatchSource:0}: Error finding container be443430a96598d207e76e5b794f8f7aa5a7ea89c93b16a5061dfe673432c076: Status 404 returned error can't find the container with id be443430a96598d207e76e5b794f8f7aa5a7ea89c93b16a5061dfe673432c076
Jul 16 21:20:43.704874 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:43.704786    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t" event={"ID":"ee959131-408e-4cde-8a52-1599d47e9173","Type":"ContainerStarted","Data":"be443430a96598d207e76e5b794f8f7aa5a7ea89c93b16a5061dfe673432c076"}
Jul 16 21:20:44.710255 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:44.710219    2572 generic.go:356] "Generic (PLEG): container finished" podID="ee959131-408e-4cde-8a52-1599d47e9173" containerID="892b784c2344e54d57beb4d426ae0ecd9bdef5610746a3a6c98aeb4d4c6f3862" exitCode=0
Jul 16 21:20:44.710740 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:44.710305    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t" event={"ID":"ee959131-408e-4cde-8a52-1599d47e9173","Type":"ContainerDied","Data":"892b784c2344e54d57beb4d426ae0ecd9bdef5610746a3a6c98aeb4d4c6f3862"}
Jul 16 21:20:44.710900 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:44.710881    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:20:46.720749 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:46.720717    2572 generic.go:356] "Generic (PLEG): container finished" podID="ee959131-408e-4cde-8a52-1599d47e9173" containerID="ca986b5ca9e4e3ce1c4fbc8de8cc3398490076be263eb42b5e455e2f3b35cbe8" exitCode=0
Jul 16 21:20:46.721154 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:46.720802    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t" event={"ID":"ee959131-408e-4cde-8a52-1599d47e9173","Type":"ContainerDied","Data":"ca986b5ca9e4e3ce1c4fbc8de8cc3398490076be263eb42b5e455e2f3b35cbe8"}
Jul 16 21:20:47.727106 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:47.727068    2572 generic.go:356] "Generic (PLEG): container finished" podID="ee959131-408e-4cde-8a52-1599d47e9173" containerID="993991a705f8002f64f9c3771ba405ee233d08f7ae70b988d401ce5163ebf3df" exitCode=0
Jul 16 21:20:47.727512 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:47.727111    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t" event={"ID":"ee959131-408e-4cde-8a52-1599d47e9173","Type":"ContainerDied","Data":"993991a705f8002f64f9c3771ba405ee233d08f7ae70b988d401ce5163ebf3df"}
Jul 16 21:20:48.858740 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:48.858710    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:48.998706 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:48.998610    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-util\") pod \"ee959131-408e-4cde-8a52-1599d47e9173\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") "
Jul 16 21:20:48.998706 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:48.998653    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-bundle\") pod \"ee959131-408e-4cde-8a52-1599d47e9173\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") "
Jul 16 21:20:48.998706 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:48.998684    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sv9h\" (UniqueName: \"kubernetes.io/projected/ee959131-408e-4cde-8a52-1599d47e9173-kube-api-access-5sv9h\") pod \"ee959131-408e-4cde-8a52-1599d47e9173\" (UID: \"ee959131-408e-4cde-8a52-1599d47e9173\") "
Jul 16 21:20:48.999485 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:48.999460    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-bundle" (OuterVolumeSpecName: "bundle") pod "ee959131-408e-4cde-8a52-1599d47e9173" (UID: "ee959131-408e-4cde-8a52-1599d47e9173"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:49.000834 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:49.000805    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee959131-408e-4cde-8a52-1599d47e9173-kube-api-access-5sv9h" (OuterVolumeSpecName: "kube-api-access-5sv9h") pod "ee959131-408e-4cde-8a52-1599d47e9173" (UID: "ee959131-408e-4cde-8a52-1599d47e9173"). InnerVolumeSpecName "kube-api-access-5sv9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:20:49.041936 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:49.041904    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-util" (OuterVolumeSpecName: "util") pod "ee959131-408e-4cde-8a52-1599d47e9173" (UID: "ee959131-408e-4cde-8a52-1599d47e9173"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:49.100251 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:49.100212    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-util\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:49.100251 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:49.100242    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee959131-408e-4cde-8a52-1599d47e9173-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:49.100251 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:49.100251    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sv9h\" (UniqueName: \"kubernetes.io/projected/ee959131-408e-4cde-8a52-1599d47e9173-kube-api-access-5sv9h\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:20:49.738046 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:49.738015    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t"
Jul 16 21:20:49.738046 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:49.738031    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afgjc6t" event={"ID":"ee959131-408e-4cde-8a52-1599d47e9173","Type":"ContainerDied","Data":"be443430a96598d207e76e5b794f8f7aa5a7ea89c93b16a5061dfe673432c076"}
Jul 16 21:20:49.738253 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:20:49.738070    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be443430a96598d207e76e5b794f8f7aa5a7ea89c93b16a5061dfe673432c076"
Jul 16 21:21:00.431391 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.431341    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"]
Jul 16 21:21:00.431807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.431654    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee959131-408e-4cde-8a52-1599d47e9173" containerName="util"
Jul 16 21:21:00.431807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.431664    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee959131-408e-4cde-8a52-1599d47e9173" containerName="util"
Jul 16 21:21:00.431807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.431677    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee959131-408e-4cde-8a52-1599d47e9173" containerName="pull"
Jul 16 21:21:00.431807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.431682    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee959131-408e-4cde-8a52-1599d47e9173" containerName="pull"
Jul 16 21:21:00.431807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.431692    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee959131-408e-4cde-8a52-1599d47e9173" containerName="extract"
Jul 16 21:21:00.431807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.431700    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee959131-408e-4cde-8a52-1599d47e9173" containerName="extract"
Jul 16 21:21:00.431807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.431747    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee959131-408e-4cde-8a52-1599d47e9173" containerName="extract"
Jul 16 21:21:00.434484 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.434461    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.437898 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.437877    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 21:21:00.438159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.438140    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xg8s6\""
Jul 16 21:21:00.438364 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.438332    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 21:21:00.438418 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.438402    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 21:21:00.438748 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.438730    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 21:21:00.455553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.455520    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"]
Jul 16 21:21:00.592656 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.592603    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb765821-10ed-48ae-8b06-2472a6b9aea9-tmp\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.592656 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.592657    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb765821-10ed-48ae-8b06-2472a6b9aea9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.592899 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.592685    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69vx5\" (UniqueName: \"kubernetes.io/projected/bb765821-10ed-48ae-8b06-2472a6b9aea9-kube-api-access-69vx5\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.592899 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.592738    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb765821-10ed-48ae-8b06-2472a6b9aea9-manifests\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.592899 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.592797    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bb765821-10ed-48ae-8b06-2472a6b9aea9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.693969 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.693872    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bb765821-10ed-48ae-8b06-2472a6b9aea9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.693969 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.693941    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb765821-10ed-48ae-8b06-2472a6b9aea9-tmp\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.693969 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.693959    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb765821-10ed-48ae-8b06-2472a6b9aea9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.693969 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.693975    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69vx5\" (UniqueName: \"kubernetes.io/projected/bb765821-10ed-48ae-8b06-2472a6b9aea9-kube-api-access-69vx5\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.694310 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.693997    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb765821-10ed-48ae-8b06-2472a6b9aea9-manifests\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.694493 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.694467    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb765821-10ed-48ae-8b06-2472a6b9aea9-manifests\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.696218 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.696189    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb765821-10ed-48ae-8b06-2472a6b9aea9-tmp\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.696493 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.696473    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb765821-10ed-48ae-8b06-2472a6b9aea9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.696553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.696489    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bb765821-10ed-48ae-8b06-2472a6b9aea9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.702310 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.702282    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69vx5\" (UniqueName: \"kubernetes.io/projected/bb765821-10ed-48ae-8b06-2472a6b9aea9-kube-api-access-69vx5\") pod \"opendatahub-operator-controller-manager-7c4b6cd7d5-947rd\" (UID: \"bb765821-10ed-48ae-8b06-2472a6b9aea9\") " pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.744412 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.744376    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:00.880113 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:00.880086    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"]
Jul 16 21:21:00.882381 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:21:00.882334    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb765821_10ed_48ae_8b06_2472a6b9aea9.slice/crio-aff68546520909cc796855e52a78483f227b6df25348bdb82873299334bed271 WatchSource:0}: Error finding container aff68546520909cc796855e52a78483f227b6df25348bdb82873299334bed271: Status 404 returned error can't find the container with id aff68546520909cc796855e52a78483f227b6df25348bdb82873299334bed271
Jul 16 21:21:01.793975 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:01.793935    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd" event={"ID":"bb765821-10ed-48ae-8b06-2472a6b9aea9","Type":"ContainerStarted","Data":"aff68546520909cc796855e52a78483f227b6df25348bdb82873299334bed271"}
Jul 16 21:21:04.809519 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:04.809487    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd" event={"ID":"bb765821-10ed-48ae-8b06-2472a6b9aea9","Type":"ContainerStarted","Data":"7c0f26c44fc1941612c22a76f53c941d9cb010e067bb1682e45943a8b2cf08c0"}
Jul 16 21:21:05.815588 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:05.815549    2572 generic.go:356] "Generic (PLEG): container finished" podID="bb765821-10ed-48ae-8b06-2472a6b9aea9" containerID="7c0f26c44fc1941612c22a76f53c941d9cb010e067bb1682e45943a8b2cf08c0" exitCode=0
Jul 16 21:21:05.815960 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:05.815654    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd" event={"ID":"bb765821-10ed-48ae-8b06-2472a6b9aea9","Type":"ContainerDied","Data":"7c0f26c44fc1941612c22a76f53c941d9cb010e067bb1682e45943a8b2cf08c0"}
Jul 16 21:21:07.826589 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:07.826554    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd" event={"ID":"bb765821-10ed-48ae-8b06-2472a6b9aea9","Type":"ContainerStarted","Data":"ad2c0a7a83b0252bca42edcf8970b676e6eb73130377bb41aa60519bac9e39f7"}
Jul 16 21:21:07.826982 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:07.826722    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:07.854383 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:07.852082    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd" podStartSLOduration=2.860600351 podStartE2EDuration="7.852061411s" totalImagesPullingTime="3.849549161s" totalInitContainerRuntime="1.141911899s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:07.850431655 +0000 UTC m=+333.361591388" watchObservedRunningTime="2026-07-16 21:21:07.852061411 +0000 UTC m=+333.363221146"
Jul 16 21:21:18.833048 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:18.833015    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c4b6cd7d5-947rd"
Jul 16 21:21:21.235876 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.235837    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"]
Jul 16 21:21:21.244471 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.244435    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.247952 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.247926    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 21:21:21.248698 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.248670    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 16 21:21:21.248804 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.248764    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-f6rp2\""
Jul 16 21:21:21.248924 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.248895    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 21:21:21.249387 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.249344    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 16 21:21:21.271151 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.271119    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"]
Jul 16 21:21:21.276108 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.276081    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-tls-certs\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.276273 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.276116    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwq8m\" (UniqueName: \"kubernetes.io/projected/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-kube-api-access-vwq8m\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.276273 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.276148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-tmp\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.377051 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.377014    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-tmp\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.377245 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.377117    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-tls-certs\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.377245 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.377151    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwq8m\" (UniqueName: \"kubernetes.io/projected/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-kube-api-access-vwq8m\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.380136 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.380100    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-tmp\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.380291 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.380272    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-tls-certs\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.389472 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.389445    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-96f2w"]
Jul 16 21:21:21.393258 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.393235    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:21.394167 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.394140    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwq8m\" (UniqueName: \"kubernetes.io/projected/0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf-kube-api-access-vwq8m\") pod \"kube-auth-proxy-796f7d4c6-snm79\" (UID: \"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf\") " pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.396465 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.396445    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 16 21:21:21.396564 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.396525    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jxvq4\""
Jul 16 21:21:21.408219 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.408193    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-96f2w"]
Jul 16 21:21:21.478246 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.478211    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ft2g\" (UniqueName: \"kubernetes.io/projected/c30d7193-e264-40d2-959e-7997f27c9760-kube-api-access-7ft2g\") pod \"odh-model-controller-7998c67dc4-96f2w\" (UID: \"c30d7193-e264-40d2-959e-7997f27c9760\") " pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:21.478440 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.478267    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c30d7193-e264-40d2-959e-7997f27c9760-cert\") pod \"odh-model-controller-7998c67dc4-96f2w\" (UID: \"c30d7193-e264-40d2-959e-7997f27c9760\") " pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:21.555445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.555407    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"
Jul 16 21:21:21.578860 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.578823    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ft2g\" (UniqueName: \"kubernetes.io/projected/c30d7193-e264-40d2-959e-7997f27c9760-kube-api-access-7ft2g\") pod \"odh-model-controller-7998c67dc4-96f2w\" (UID: \"c30d7193-e264-40d2-959e-7997f27c9760\") " pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:21.579022 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.578904    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c30d7193-e264-40d2-959e-7997f27c9760-cert\") pod \"odh-model-controller-7998c67dc4-96f2w\" (UID: \"c30d7193-e264-40d2-959e-7997f27c9760\") " pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:21.579098 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:21:21.579040    2572 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 16 21:21:21.579210 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:21:21.579110    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c30d7193-e264-40d2-959e-7997f27c9760-cert podName:c30d7193-e264-40d2-959e-7997f27c9760 nodeName:}" failed. No retries permitted until 2026-07-16 21:21:22.07909059 +0000 UTC m=+347.590250319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c30d7193-e264-40d2-959e-7997f27c9760-cert") pod "odh-model-controller-7998c67dc4-96f2w" (UID: "c30d7193-e264-40d2-959e-7997f27c9760") : secret "odh-model-controller-webhook-cert" not found
Jul 16 21:21:21.593606 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.593578    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ft2g\" (UniqueName: \"kubernetes.io/projected/c30d7193-e264-40d2-959e-7997f27c9760-kube-api-access-7ft2g\") pod \"odh-model-controller-7998c67dc4-96f2w\" (UID: \"c30d7193-e264-40d2-959e-7997f27c9760\") " pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:21.710923 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.710892    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-796f7d4c6-snm79"]
Jul 16 21:21:21.715178 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:21:21.715134    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac3ed4d_04a1_4a1c_89f6_3b7ced1d77cf.slice/crio-3692d97b19bb0d42cf422c13d4e5fddafc0629a123787a814b0d4d999802fca5 WatchSource:0}: Error finding container 3692d97b19bb0d42cf422c13d4e5fddafc0629a123787a814b0d4d999802fca5: Status 404 returned error can't find the container with id 3692d97b19bb0d42cf422c13d4e5fddafc0629a123787a814b0d4d999802fca5
Jul 16 21:21:21.891832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:21.891739    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79" event={"ID":"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf","Type":"ContainerStarted","Data":"3692d97b19bb0d42cf422c13d4e5fddafc0629a123787a814b0d4d999802fca5"}
Jul 16 21:21:22.083832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.083794    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c30d7193-e264-40d2-959e-7997f27c9760-cert\") pod \"odh-model-controller-7998c67dc4-96f2w\" (UID: \"c30d7193-e264-40d2-959e-7997f27c9760\") " pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:22.086303 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.086275    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c30d7193-e264-40d2-959e-7997f27c9760-cert\") pod \"odh-model-controller-7998c67dc4-96f2w\" (UID: \"c30d7193-e264-40d2-959e-7997f27c9760\") " pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:22.310291 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.310242    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:22.474458 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.474428    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-96f2w"]
Jul 16 21:21:22.477853 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:21:22.477820    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30d7193_e264_40d2_959e_7997f27c9760.slice/crio-b91ffe24fe4efb868c0e93a2b55707cb772cd28937d6b9bd22720c050b15c501 WatchSource:0}: Error finding container b91ffe24fe4efb868c0e93a2b55707cb772cd28937d6b9bd22720c050b15c501: Status 404 returned error can't find the container with id b91ffe24fe4efb868c0e93a2b55707cb772cd28937d6b9bd22720c050b15c501
Jul 16 21:21:22.509633 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.509591    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n4s9m"]
Jul 16 21:21:22.514709 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.514680    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:22.517270 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.517183    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 16 21:21:22.517270 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.517261    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-6659m\""
Jul 16 21:21:22.525372 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.525327    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n4s9m"]
Jul 16 21:21:22.588945 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.588846    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqc9f\" (UniqueName: \"kubernetes.io/projected/bbee227a-765d-4d7d-87fd-ccad2a11870e-kube-api-access-mqc9f\") pod \"model-serving-api-54d89bd5fc-n4s9m\" (UID: \"bbee227a-765d-4d7d-87fd-ccad2a11870e\") " pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:22.589103 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.588967    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbee227a-765d-4d7d-87fd-ccad2a11870e-tls-certs\") pod \"model-serving-api-54d89bd5fc-n4s9m\" (UID: \"bbee227a-765d-4d7d-87fd-ccad2a11870e\") " pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:22.689634 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.689601    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbee227a-765d-4d7d-87fd-ccad2a11870e-tls-certs\") pod \"model-serving-api-54d89bd5fc-n4s9m\" (UID: \"bbee227a-765d-4d7d-87fd-ccad2a11870e\") " pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:22.689826 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.689668    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqc9f\" (UniqueName: \"kubernetes.io/projected/bbee227a-765d-4d7d-87fd-ccad2a11870e-kube-api-access-mqc9f\") pod \"model-serving-api-54d89bd5fc-n4s9m\" (UID: \"bbee227a-765d-4d7d-87fd-ccad2a11870e\") " pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:22.692456 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.692424    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbee227a-765d-4d7d-87fd-ccad2a11870e-tls-certs\") pod \"model-serving-api-54d89bd5fc-n4s9m\" (UID: \"bbee227a-765d-4d7d-87fd-ccad2a11870e\") " pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:22.698837 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.698781    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqc9f\" (UniqueName: \"kubernetes.io/projected/bbee227a-765d-4d7d-87fd-ccad2a11870e-kube-api-access-mqc9f\") pod \"model-serving-api-54d89bd5fc-n4s9m\" (UID: \"bbee227a-765d-4d7d-87fd-ccad2a11870e\") " pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:22.828417 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.828382    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:22.910381 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.909507    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-96f2w" event={"ID":"c30d7193-e264-40d2-959e-7997f27c9760","Type":"ContainerStarted","Data":"b91ffe24fe4efb868c0e93a2b55707cb772cd28937d6b9bd22720c050b15c501"}
Jul 16 21:21:22.983387 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:22.983342    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-n4s9m"]
Jul 16 21:21:22.986245 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:21:22.986207    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbee227a_765d_4d7d_87fd_ccad2a11870e.slice/crio-8fc9149c1ba853047ebd9b78723f704c1213d9df05db9069c171948f97cf7c63 WatchSource:0}: Error finding container 8fc9149c1ba853047ebd9b78723f704c1213d9df05db9069c171948f97cf7c63: Status 404 returned error can't find the container with id 8fc9149c1ba853047ebd9b78723f704c1213d9df05db9069c171948f97cf7c63
Jul 16 21:21:23.919878 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:23.919817    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m" event={"ID":"bbee227a-765d-4d7d-87fd-ccad2a11870e","Type":"ContainerStarted","Data":"8fc9149c1ba853047ebd9b78723f704c1213d9df05db9069c171948f97cf7c63"}
Jul 16 21:21:26.834020 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.833987    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"]
Jul 16 21:21:26.837521 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.837499    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.840122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.840100    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-77xbw\""
Jul 16 21:21:26.840232 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.840107    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 21:21:26.854482 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.854452    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"]
Jul 16 21:21:26.931951 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.931913    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4894a3a-2c73-44cf-8995-a78743186669-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.932133 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.931968    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.932133 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.932072    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.932133 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.932113    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.932255 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.932154    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.932255 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.932196    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4894a3a-2c73-44cf-8995-a78743186669-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.932255 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.932216    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.932399 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.932276    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4894a3a-2c73-44cf-8995-a78743186669-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.932399 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.932296    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5w5s\" (UniqueName: \"kubernetes.io/projected/b4894a3a-2c73-44cf-8995-a78743186669-kube-api-access-t5w5s\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:26.937449 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.937415    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-96f2w" event={"ID":"c30d7193-e264-40d2-959e-7997f27c9760","Type":"ContainerStarted","Data":"bcaad96a44f06b95d7061cc62973b685e28c57a18c347fdb78c8b62e5d2f9faf"}
Jul 16 21:21:26.937622 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.937539    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:26.938922 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.938896    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m" event={"ID":"bbee227a-765d-4d7d-87fd-ccad2a11870e","Type":"ContainerStarted","Data":"90547cd653ab98e7e1e49c452ba65dc882a4b7bf244aa1d2391de4fa4fbb3a61"}
Jul 16 21:21:26.939061 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.938953    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:21:26.940322 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.940292    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79" event={"ID":"0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf","Type":"ContainerStarted","Data":"8cd45b87dd65d54810461909d564c6c985bbf97ac76d420cc4992978655fec5b"}
Jul 16 21:21:26.955587 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.955533    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-96f2w" podStartSLOduration=2.3645608940000002 podStartE2EDuration="5.95551505s" totalImagesPullingTime="3.590954156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:26.953701844 +0000 UTC m=+352.464861578" watchObservedRunningTime="2026-07-16 21:21:26.95551505 +0000 UTC m=+352.466674762"
Jul 16 21:21:26.971471 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.971421    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m" podStartSLOduration=1.8919368460000001 podStartE2EDuration="4.971403149s" totalImagesPullingTime="3.079466303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:26.971060642 +0000 UTC m=+352.482220388" watchObservedRunningTime="2026-07-16 21:21:26.971403149 +0000 UTC m=+352.482562882"
Jul 16 21:21:26.991450 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:26.991387    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-796f7d4c6-snm79" podStartSLOduration=1.643843406 podStartE2EDuration="5.991366883s" totalImagesPullingTime="4.347523477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:26.989540092 +0000 UTC m=+352.500699828" watchObservedRunningTime="2026-07-16 21:21:26.991366883 +0000 UTC m=+352.502526616"
Jul 16 21:21:27.033198 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033152    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033375 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033253    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033375 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033304    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4894a3a-2c73-44cf-8995-a78743186669-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033519 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033395    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033519 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033449    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4894a3a-2c73-44cf-8995-a78743186669-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033519 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033480    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5w5s\" (UniqueName: \"kubernetes.io/projected/b4894a3a-2c73-44cf-8995-a78743186669-kube-api-access-t5w5s\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033667 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033534    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4894a3a-2c73-44cf-8995-a78743186669-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033667 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033612    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033667 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033629    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033873 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033678    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.033949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.033925    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.034073 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.034045    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4894a3a-2c73-44cf-8995-a78743186669-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.034251 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.034222    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.034453 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.034432    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.036045 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.036019    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4894a3a-2c73-44cf-8995-a78743186669-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.036045 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.036034    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4894a3a-2c73-44cf-8995-a78743186669-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.046782 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.046755    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5w5s\" (UniqueName: \"kubernetes.io/projected/b4894a3a-2c73-44cf-8995-a78743186669-kube-api-access-t5w5s\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.047223 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.047200    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4894a3a-2c73-44cf-8995-a78743186669-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4q9wpq\" (UID: \"b4894a3a-2c73-44cf-8995-a78743186669\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.147439 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.147335    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:27.291630 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.291602    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"]
Jul 16 21:21:27.293210 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:21:27.293185    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4894a3a_2c73_44cf_8995_a78743186669.slice/crio-6c609a48e0a761e5d3d9c59a6e4440f879c1011a8d003d2e7bca1001ac0edcbf WatchSource:0}: Error finding container 6c609a48e0a761e5d3d9c59a6e4440f879c1011a8d003d2e7bca1001ac0edcbf: Status 404 returned error can't find the container with id 6c609a48e0a761e5d3d9c59a6e4440f879c1011a8d003d2e7bca1001ac0edcbf
Jul 16 21:21:27.946677 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:27.946642    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq" event={"ID":"b4894a3a-2c73-44cf-8995-a78743186669","Type":"ContainerStarted","Data":"6c609a48e0a761e5d3d9c59a6e4440f879c1011a8d003d2e7bca1001ac0edcbf"}
Jul 16 21:21:28.594854 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.594814    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"]
Jul 16 21:21:28.598739 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.598713    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.602344 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.602319    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 16 21:21:28.602344 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.602320    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 16 21:21:28.602554 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.602320    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-72vp7\""
Jul 16 21:21:28.602554 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.602457    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 16 21:21:28.618377 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.618335    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"]
Jul 16 21:21:28.647562 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.647521    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8kq4\" (UniqueName: \"kubernetes.io/projected/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-kube-api-access-v8kq4\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.647739 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.647578    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.647739 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.647620    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.647739 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.647683    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.647739 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.647734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.647958 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.647777    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.647958 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.647835    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.748510 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.748473    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.748713 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.748557    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8kq4\" (UniqueName: \"kubernetes.io/projected/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-kube-api-access-v8kq4\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.748713 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.748593    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.748713 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.748627    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.748713 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.748663    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.748713 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.748692    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.748982 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.748730    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.749515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.749473    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.751974 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.751918    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.752125 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.751971    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.752125 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.752011    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.753096 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.753051    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.765075 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.765046    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.765241 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.765079    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8kq4\" (UniqueName: \"kubernetes.io/projected/4cdbd363-83f5-4337-b1d3-8b2f5617cd2e-kube-api-access-v8kq4\") pod \"istiod-openshift-gateway-94bb8fbfd-hjsck\" (UID: \"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:28.912132 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:28.911802    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:29.078682 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:29.078613    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"]
Jul 16 21:21:30.016725 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:21:30.016691    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cdbd363_83f5_4337_b1d3_8b2f5617cd2e.slice/crio-6eff4e261962dbc60523cdd2ed00db59bfbb333c38d3aa42c258463b149f6700 WatchSource:0}: Error finding container 6eff4e261962dbc60523cdd2ed00db59bfbb333c38d3aa42c258463b149f6700: Status 404 returned error can't find the container with id 6eff4e261962dbc60523cdd2ed00db59bfbb333c38d3aa42c258463b149f6700
Jul 16 21:21:30.096152 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:30.096115    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:21:30.096481 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:30.096190    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:21:30.096481 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:30.096215    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:21:30.968223 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:30.968180    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq" event={"ID":"b4894a3a-2c73-44cf-8995-a78743186669","Type":"ContainerStarted","Data":"0750c52b0ec5678e84393b8e3c151df3225ae8b4e66071f21eff16a5653de6bb"}
Jul 16 21:21:30.969604 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:30.969574    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck" event={"ID":"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e","Type":"ContainerStarted","Data":"6eff4e261962dbc60523cdd2ed00db59bfbb333c38d3aa42c258463b149f6700"}
Jul 16 21:21:31.054156 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.054094    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq" podStartSLOduration=2.25394054 podStartE2EDuration="5.054076872s" totalImagesPullingTime="2.800136332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:31.042934428 +0000 UTC m=+356.554094163" watchObservedRunningTime="2026-07-16 21:21:31.054076872 +0000 UTC m=+356.565236606"
Jul 16 21:21:31.148166 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.148129    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:31.154374 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.154322    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:31.939580 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.939545    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"]
Jul 16 21:21:31.946792 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.946762    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:31.950792 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.950767    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 16 21:21:31.950941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.950896    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-r2dkn\""
Jul 16 21:21:31.960648 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.960618    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"]
Jul 16 21:21:31.972088 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.972054    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"]
Jul 16 21:21:31.976505 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.975814    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:31.976787 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.976757    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:31.977823 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.977797    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4q9wpq"
Jul 16 21:21:31.980060 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.979997    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 16 21:21:31.980060 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.980026    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-dldnn\""
Jul 16 21:21:31.997038 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:31.997008    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"]
Jul 16 21:21:32.081163 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.081120    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82f32722-26a1-4261-a07d-a07ead89c7c5-cert\") pod \"kserve-controller-manager-d6f4b65b8-f4k5g\" (UID: \"82f32722-26a1-4261-a07d-a07ead89c7c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:32.081344 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.081228    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rs6k\" (UniqueName: \"kubernetes.io/projected/82f32722-26a1-4261-a07d-a07ead89c7c5-kube-api-access-2rs6k\") pod \"kserve-controller-manager-d6f4b65b8-f4k5g\" (UID: \"82f32722-26a1-4261-a07d-a07ead89c7c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:32.081444 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.081338    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2p88\" (UniqueName: \"kubernetes.io/projected/d7626221-53db-424f-abfb-958a90cc6f03-kube-api-access-z2p88\") pod \"llmisvc-controller-manager-777c6bd7d5-dcswj\" (UID: \"d7626221-53db-424f-abfb-958a90cc6f03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:32.081444 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.081402    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7626221-53db-424f-abfb-958a90cc6f03-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dcswj\" (UID: \"d7626221-53db-424f-abfb-958a90cc6f03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:32.182685 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.182649    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2p88\" (UniqueName: \"kubernetes.io/projected/d7626221-53db-424f-abfb-958a90cc6f03-kube-api-access-z2p88\") pod \"llmisvc-controller-manager-777c6bd7d5-dcswj\" (UID: \"d7626221-53db-424f-abfb-958a90cc6f03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:32.183141 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.182703    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7626221-53db-424f-abfb-958a90cc6f03-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dcswj\" (UID: \"d7626221-53db-424f-abfb-958a90cc6f03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:32.183141 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.182758    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82f32722-26a1-4261-a07d-a07ead89c7c5-cert\") pod \"kserve-controller-manager-d6f4b65b8-f4k5g\" (UID: \"82f32722-26a1-4261-a07d-a07ead89c7c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:32.183141 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.182800    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rs6k\" (UniqueName: \"kubernetes.io/projected/82f32722-26a1-4261-a07d-a07ead89c7c5-kube-api-access-2rs6k\") pod \"kserve-controller-manager-d6f4b65b8-f4k5g\" (UID: \"82f32722-26a1-4261-a07d-a07ead89c7c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:32.183141 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:21:32.182815    2572 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 16 21:21:32.183141 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:21:32.182877    2572 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 16 21:21:32.183141 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:21:32.182896    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7626221-53db-424f-abfb-958a90cc6f03-cert podName:d7626221-53db-424f-abfb-958a90cc6f03 nodeName:}" failed. No retries permitted until 2026-07-16 21:21:32.682877083 +0000 UTC m=+358.194036798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d7626221-53db-424f-abfb-958a90cc6f03-cert") pod "llmisvc-controller-manager-777c6bd7d5-dcswj" (UID: "d7626221-53db-424f-abfb-958a90cc6f03") : secret "llmisvc-webhook-server-cert" not found
Jul 16 21:21:32.183141 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:21:32.182933    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82f32722-26a1-4261-a07d-a07ead89c7c5-cert podName:82f32722-26a1-4261-a07d-a07ead89c7c5 nodeName:}" failed. No retries permitted until 2026-07-16 21:21:32.682914859 +0000 UTC m=+358.194074586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/82f32722-26a1-4261-a07d-a07ead89c7c5-cert") pod "kserve-controller-manager-d6f4b65b8-f4k5g" (UID: "82f32722-26a1-4261-a07d-a07ead89c7c5") : secret "kserve-webhook-server-cert" not found
Jul 16 21:21:32.207027 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.206955    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rs6k\" (UniqueName: \"kubernetes.io/projected/82f32722-26a1-4261-a07d-a07ead89c7c5-kube-api-access-2rs6k\") pod \"kserve-controller-manager-d6f4b65b8-f4k5g\" (UID: \"82f32722-26a1-4261-a07d-a07ead89c7c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:32.209801 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.209772    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2p88\" (UniqueName: \"kubernetes.io/projected/d7626221-53db-424f-abfb-958a90cc6f03-kube-api-access-z2p88\") pod \"llmisvc-controller-manager-777c6bd7d5-dcswj\" (UID: \"d7626221-53db-424f-abfb-958a90cc6f03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:32.530083 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.530037    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:21:32.530208 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.530127    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:21:32.687860 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.687820    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7626221-53db-424f-abfb-958a90cc6f03-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dcswj\" (UID: \"d7626221-53db-424f-abfb-958a90cc6f03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:32.688033 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.687871    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82f32722-26a1-4261-a07d-a07ead89c7c5-cert\") pod \"kserve-controller-manager-d6f4b65b8-f4k5g\" (UID: \"82f32722-26a1-4261-a07d-a07ead89c7c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:32.690444 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.690418    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7626221-53db-424f-abfb-958a90cc6f03-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-dcswj\" (UID: \"d7626221-53db-424f-abfb-958a90cc6f03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:32.690550 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.690501    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/82f32722-26a1-4261-a07d-a07ead89c7c5-cert\") pod \"kserve-controller-manager-d6f4b65b8-f4k5g\" (UID: \"82f32722-26a1-4261-a07d-a07ead89c7c5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:32.859781 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.859683    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:32.888545 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.888513    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:32.987762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.987696    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck" event={"ID":"4cdbd363-83f5-4337-b1d3-8b2f5617cd2e","Type":"ContainerStarted","Data":"331bfb704e6f3b8e8c5d0aebb182280e45c9424cfaca250fffc21f8d5ca76f6d"}
Jul 16 21:21:32.988032 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:32.987987    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:33.015989 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:33.015850    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"]
Jul 16 21:21:33.017702 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:21:33.017666    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f32722_26a1_4261_a07d_a07ead89c7c5.slice/crio-c30a3d56de6d9db98e656a03f9f8eb8bd69964eabcd60a229528077ce9d08109 WatchSource:0}: Error finding container c30a3d56de6d9db98e656a03f9f8eb8bd69964eabcd60a229528077ce9d08109: Status 404 returned error can't find the container with id c30a3d56de6d9db98e656a03f9f8eb8bd69964eabcd60a229528077ce9d08109
Jul 16 21:21:33.018264 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:33.018222    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck" podStartSLOduration=2.507332072 podStartE2EDuration="5.018204093s" totalImagesPullingTime="2.510872021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:33.013067818 +0000 UTC m=+358.524227550" watchObservedRunningTime="2026-07-16 21:21:33.018204093 +0000 UTC m=+358.529363827"
Jul 16 21:21:33.051701 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:33.051674    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"]
Jul 16 21:21:33.053622 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:21:33.053592    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd7626221_53db_424f_abfb_958a90cc6f03.slice/crio-f58bd437f79b46f75e4368317d493846f3ed7090fe4ffa9f8f7217c70fea78b6 WatchSource:0}: Error finding container f58bd437f79b46f75e4368317d493846f3ed7090fe4ffa9f8f7217c70fea78b6: Status 404 returned error can't find the container with id f58bd437f79b46f75e4368317d493846f3ed7090fe4ffa9f8f7217c70fea78b6
Jul 16 21:21:33.995878 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:33.995828    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g" event={"ID":"82f32722-26a1-4261-a07d-a07ead89c7c5","Type":"ContainerStarted","Data":"c30a3d56de6d9db98e656a03f9f8eb8bd69964eabcd60a229528077ce9d08109"}
Jul 16 21:21:33.997620 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:33.997587    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj" event={"ID":"d7626221-53db-424f-abfb-958a90cc6f03","Type":"ContainerStarted","Data":"f58bd437f79b46f75e4368317d493846f3ed7090fe4ffa9f8f7217c70fea78b6"}
Jul 16 21:21:34.000230 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:34.000204    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-hjsck"
Jul 16 21:21:37.016084 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:37.016043    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g" event={"ID":"82f32722-26a1-4261-a07d-a07ead89c7c5","Type":"ContainerStarted","Data":"fae08ba3592c55ca88d9a1c5f46e4f58c9db856e498f30c8f40e85eb56338b8b"}
Jul 16 21:21:37.016541 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:37.016217    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:21:37.017576 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:37.017545    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj" event={"ID":"d7626221-53db-424f-abfb-958a90cc6f03","Type":"ContainerStarted","Data":"8075d1cdd225b589e730c6567a6a48d49b74b53034e7e2ae54ce8cae77ac3a2c"}
Jul 16 21:21:37.017643 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:37.017606    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:21:37.035724 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:37.035677    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g" podStartSLOduration=2.759121048 podStartE2EDuration="6.035662675s" totalImagesPullingTime="3.276541627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:37.034023433 +0000 UTC m=+362.545183168" watchObservedRunningTime="2026-07-16 21:21:37.035662675 +0000 UTC m=+362.546822409"
Jul 16 21:21:37.052807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:37.052756    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj" podStartSLOduration=2.807206645 podStartE2EDuration="6.052738751s" totalImagesPullingTime="3.245532106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:37.051752832 +0000 UTC m=+362.562912575" watchObservedRunningTime="2026-07-16 21:21:37.052738751 +0000 UTC m=+362.563898485"
Jul 16 21:21:37.949271 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:37.949241    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-96f2w"
Jul 16 21:21:37.951301 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:21:37.951273    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-n4s9m"
Jul 16 21:22:08.025988 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:08.025953    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-dcswj"
Jul 16 21:22:08.029548 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:08.029525    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f4k5g"
Jul 16 21:22:37.198978 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.198938    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"]
Jul 16 21:22:37.201704 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.201684    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.204559 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.204533    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 21:22:37.204671 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.204534    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 21:22:37.205512 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.205496    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-smnkp\""
Jul 16 21:22:37.212134 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.212106    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"]
Jul 16 21:22:37.348167 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.348134    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.348167 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.348181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.348442 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.348287    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxq2p\" (UniqueName: \"kubernetes.io/projected/61654333-ca77-4a1e-822f-8a10d1b3ad73-kube-api-access-gxq2p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.449240 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.449129    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxq2p\" (UniqueName: \"kubernetes.io/projected/61654333-ca77-4a1e-822f-8a10d1b3ad73-kube-api-access-gxq2p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.449445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.449271    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.449445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.449336    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.449703 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.449683    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.449747 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.449713    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.463973 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.463940    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxq2p\" (UniqueName: \"kubernetes.io/projected/61654333-ca77-4a1e-822f-8a10d1b3ad73-kube-api-access-gxq2p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.512071 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.512033    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:37.651441 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:37.651397    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"]
Jul 16 21:22:37.657660 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:22:37.657630    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61654333_ca77_4a1e_822f_8a10d1b3ad73.slice/crio-9b403aefffd2d0cda8d6365b9344e9797674e488cbdb143e2a2bce6ed8b1259b WatchSource:0}: Error finding container 9b403aefffd2d0cda8d6365b9344e9797674e488cbdb143e2a2bce6ed8b1259b: Status 404 returned error can't find the container with id 9b403aefffd2d0cda8d6365b9344e9797674e488cbdb143e2a2bce6ed8b1259b
Jul 16 21:22:38.013405 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.013324    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"]
Jul 16 21:22:38.016004 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.015987    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.026467 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.026437    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"]
Jul 16 21:22:38.154682 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.154645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.154863 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.154702    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chp9j\" (UniqueName: \"kubernetes.io/projected/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-kube-api-access-chp9j\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.154863 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.154762    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.255498 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.255459    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.255972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.255516    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chp9j\" (UniqueName: \"kubernetes.io/projected/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-kube-api-access-chp9j\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.255972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.255637    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.255972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.255943    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.255972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.255952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.264696 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.264640    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chp9j\" (UniqueName: \"kubernetes.io/projected/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-kube-api-access-chp9j\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.324431 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.324391    2572 generic.go:356] "Generic (PLEG): container finished" podID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerID="b338003f88a045719ef19cf5cf41f975d6601abd55fd119000e5238645b9837c" exitCode=0
Jul 16 21:22:38.324593 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.324485    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h" event={"ID":"61654333-ca77-4a1e-822f-8a10d1b3ad73","Type":"ContainerDied","Data":"b338003f88a045719ef19cf5cf41f975d6601abd55fd119000e5238645b9837c"}
Jul 16 21:22:38.324593 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.324537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h" event={"ID":"61654333-ca77-4a1e-822f-8a10d1b3ad73","Type":"ContainerStarted","Data":"9b403aefffd2d0cda8d6365b9344e9797674e488cbdb143e2a2bce6ed8b1259b"}
Jul 16 21:22:38.325377 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.325297    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:38.405580 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.405547    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"]
Jul 16 21:22:38.408631 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.408609    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.419889 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.419860    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"]
Jul 16 21:22:38.474315 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.474196    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"]
Jul 16 21:22:38.476559 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:22:38.476528    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd963ca4_8972_4a62_840c_d4cbdaa88cf7.slice/crio-5185c63e293d3541798210e6e5c97e06b57de6080ed26fa83587c1856462d550 WatchSource:0}: Error finding container 5185c63e293d3541798210e6e5c97e06b57de6080ed26fa83587c1856462d550: Status 404 returned error can't find the container with id 5185c63e293d3541798210e6e5c97e06b57de6080ed26fa83587c1856462d550
Jul 16 21:22:38.558851 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.558812    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.559033 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.558866    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmlxs\" (UniqueName: \"kubernetes.io/projected/b6d8de27-f4d2-48c6-8e21-28883d5025cd-kube-api-access-jmlxs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.559077 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.559021    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.660164 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.660125    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.660164 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.660174    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmlxs\" (UniqueName: \"kubernetes.io/projected/b6d8de27-f4d2-48c6-8e21-28883d5025cd-kube-api-access-jmlxs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.660505 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.660220    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.660665 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.660640    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.660665 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.660659    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.670164 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.670134    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmlxs\" (UniqueName: \"kubernetes.io/projected/b6d8de27-f4d2-48c6-8e21-28883d5025cd-kube-api-access-jmlxs\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.720058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.720020    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:38.822767 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.822731    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"]
Jul 16 21:22:38.826547 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.826520    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:38.840326 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.840295    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"]
Jul 16 21:22:38.872241 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.872211    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"]
Jul 16 21:22:38.873894 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:22:38.873865    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d8de27_f4d2_48c6_8e21_28883d5025cd.slice/crio-d7a33366a3376a59fb02136fc22f4fa8affeda86ef8ff54ad30930fe829d2ee7 WatchSource:0}: Error finding container d7a33366a3376a59fb02136fc22f4fa8affeda86ef8ff54ad30930fe829d2ee7: Status 404 returned error can't find the container with id d7a33366a3376a59fb02136fc22f4fa8affeda86ef8ff54ad30930fe829d2ee7
Jul 16 21:22:38.963118 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.963066    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:38.963118 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.963114    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:38.963466 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:38.963204    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfpgn\" (UniqueName: \"kubernetes.io/projected/fe78c421-68ae-4b06-983c-f3ba2bd24747-kube-api-access-xfpgn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:38.992482 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:22:38.992447    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d8de27_f4d2_48c6_8e21_28883d5025cd.slice/crio-1530ce84948762b7c9999ff740496a091c9ff4a52b825e02e6e3a3f381854967.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d8de27_f4d2_48c6_8e21_28883d5025cd.slice/crio-conmon-1530ce84948762b7c9999ff740496a091c9ff4a52b825e02e6e3a3f381854967.scope\": RecentStats: unable to find data in memory cache]"
Jul 16 21:22:38.992636 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:22:38.992522    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d8de27_f4d2_48c6_8e21_28883d5025cd.slice/crio-conmon-1530ce84948762b7c9999ff740496a091c9ff4a52b825e02e6e3a3f381854967.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d8de27_f4d2_48c6_8e21_28883d5025cd.slice/crio-1530ce84948762b7c9999ff740496a091c9ff4a52b825e02e6e3a3f381854967.scope\": RecentStats: unable to find data in memory cache]"
Jul 16 21:22:39.064753 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.064715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfpgn\" (UniqueName: \"kubernetes.io/projected/fe78c421-68ae-4b06-983c-f3ba2bd24747-kube-api-access-xfpgn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:39.064926 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.064835    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:39.064926 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.064867    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:39.065267 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.065243    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:39.065267 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.065263    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:39.075841 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.075809    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfpgn\" (UniqueName: \"kubernetes.io/projected/fe78c421-68ae-4b06-983c-f3ba2bd24747-kube-api-access-xfpgn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:39.137612 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.137577    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:39.306844 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.306813    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"]
Jul 16 21:22:39.331753 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.331669    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerID="1530ce84948762b7c9999ff740496a091c9ff4a52b825e02e6e3a3f381854967" exitCode=0
Jul 16 21:22:39.331753 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.331710    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8" event={"ID":"b6d8de27-f4d2-48c6-8e21-28883d5025cd","Type":"ContainerDied","Data":"1530ce84948762b7c9999ff740496a091c9ff4a52b825e02e6e3a3f381854967"}
Jul 16 21:22:39.331957 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.331772    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8" event={"ID":"b6d8de27-f4d2-48c6-8e21-28883d5025cd","Type":"ContainerStarted","Data":"d7a33366a3376a59fb02136fc22f4fa8affeda86ef8ff54ad30930fe829d2ee7"}
Jul 16 21:22:39.333629 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.333608    2572 generic.go:356] "Generic (PLEG): container finished" podID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerID="2ce4608cdfbfd8abe7840f17d3e34bcf751e3110218095d2d4574bde9d570e59" exitCode=0
Jul 16 21:22:39.333735 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.333691    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h" event={"ID":"61654333-ca77-4a1e-822f-8a10d1b3ad73","Type":"ContainerDied","Data":"2ce4608cdfbfd8abe7840f17d3e34bcf751e3110218095d2d4574bde9d570e59"}
Jul 16 21:22:39.335409 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.335385    2572 generic.go:356] "Generic (PLEG): container finished" podID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerID="f8d3529df1d540e857d418b544dee6b24991a048f55218102d0de7672210b391" exitCode=0
Jul 16 21:22:39.335507 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.335425    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw" event={"ID":"bd963ca4-8972-4a62-840c-d4cbdaa88cf7","Type":"ContainerDied","Data":"f8d3529df1d540e857d418b544dee6b24991a048f55218102d0de7672210b391"}
Jul 16 21:22:39.335507 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:39.335469    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw" event={"ID":"bd963ca4-8972-4a62-840c-d4cbdaa88cf7","Type":"ContainerStarted","Data":"5185c63e293d3541798210e6e5c97e06b57de6080ed26fa83587c1856462d550"}
Jul 16 21:22:39.341766 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:22:39.341740    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe78c421_68ae_4b06_983c_f3ba2bd24747.slice/crio-145cb57aa36892b18925d53832fd6eb0997c029c10e96b81c8500fb94f862be0 WatchSource:0}: Error finding container 145cb57aa36892b18925d53832fd6eb0997c029c10e96b81c8500fb94f862be0: Status 404 returned error can't find the container with id 145cb57aa36892b18925d53832fd6eb0997c029c10e96b81c8500fb94f862be0
Jul 16 21:22:40.342539 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.342504    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerID="e23ca11eb01ed905eb5c81ba3f08f2ecc8db3e44121541d36cda7ec09250f58c" exitCode=0
Jul 16 21:22:40.342941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.342584    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8" event={"ID":"b6d8de27-f4d2-48c6-8e21-28883d5025cd","Type":"ContainerDied","Data":"e23ca11eb01ed905eb5c81ba3f08f2ecc8db3e44121541d36cda7ec09250f58c"}
Jul 16 21:22:40.346412 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.346220    2572 generic.go:356] "Generic (PLEG): container finished" podID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerID="39594b07e107d94be22a02cdc88ae1bc77eb5578f1db0c2d98bf42336d01736c" exitCode=0
Jul 16 21:22:40.346547 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.346404    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h" event={"ID":"61654333-ca77-4a1e-822f-8a10d1b3ad73","Type":"ContainerDied","Data":"39594b07e107d94be22a02cdc88ae1bc77eb5578f1db0c2d98bf42336d01736c"}
Jul 16 21:22:40.352858 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.352827    2572 generic.go:356] "Generic (PLEG): container finished" podID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerID="8089f1549758869dab2e4b4109076ccab602f999f58d95b1444a2f3c2864349d" exitCode=0
Jul 16 21:22:40.352966 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.352903    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw" event={"ID":"bd963ca4-8972-4a62-840c-d4cbdaa88cf7","Type":"ContainerDied","Data":"8089f1549758869dab2e4b4109076ccab602f999f58d95b1444a2f3c2864349d"}
Jul 16 21:22:40.354867 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.354842    2572 generic.go:356] "Generic (PLEG): container finished" podID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerID="739a83135599db5c4aee7cb3bfba050f4ba911786e959a883997b774048ec5fc" exitCode=0
Jul 16 21:22:40.354957 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.354891    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj" event={"ID":"fe78c421-68ae-4b06-983c-f3ba2bd24747","Type":"ContainerDied","Data":"739a83135599db5c4aee7cb3bfba050f4ba911786e959a883997b774048ec5fc"}
Jul 16 21:22:40.354957 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:40.354936    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj" event={"ID":"fe78c421-68ae-4b06-983c-f3ba2bd24747","Type":"ContainerStarted","Data":"145cb57aa36892b18925d53832fd6eb0997c029c10e96b81c8500fb94f862be0"}
Jul 16 21:22:41.362548 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.362454    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerID="f156e8da8db34e17754e9ca5fee770711582ec530b1ce19484497ef4c6e86173" exitCode=0
Jul 16 21:22:41.362913 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.362549    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8" event={"ID":"b6d8de27-f4d2-48c6-8e21-28883d5025cd","Type":"ContainerDied","Data":"f156e8da8db34e17754e9ca5fee770711582ec530b1ce19484497ef4c6e86173"}
Jul 16 21:22:41.364658 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.364636    2572 generic.go:356] "Generic (PLEG): container finished" podID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerID="71fdac296bb41fbd82d73f60ae2107d31146d66be3fd84e840242c859bd14b8b" exitCode=0
Jul 16 21:22:41.364745 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.364702    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw" event={"ID":"bd963ca4-8972-4a62-840c-d4cbdaa88cf7","Type":"ContainerDied","Data":"71fdac296bb41fbd82d73f60ae2107d31146d66be3fd84e840242c859bd14b8b"}
Jul 16 21:22:41.366365 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.366329    2572 generic.go:356] "Generic (PLEG): container finished" podID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerID="ed1f5386dc3d79da926ca3c7e994786d0156655f878a665abeb95ac98719eb49" exitCode=0
Jul 16 21:22:41.366472 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.366386    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj" event={"ID":"fe78c421-68ae-4b06-983c-f3ba2bd24747","Type":"ContainerDied","Data":"ed1f5386dc3d79da926ca3c7e994786d0156655f878a665abeb95ac98719eb49"}
Jul 16 21:22:41.560602 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.560576    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:41.688443 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.688339    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-bundle\") pod \"61654333-ca77-4a1e-822f-8a10d1b3ad73\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") "
Jul 16 21:22:41.688601 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.688466    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxq2p\" (UniqueName: \"kubernetes.io/projected/61654333-ca77-4a1e-822f-8a10d1b3ad73-kube-api-access-gxq2p\") pod \"61654333-ca77-4a1e-822f-8a10d1b3ad73\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") "
Jul 16 21:22:41.688601 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.688495    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-util\") pod \"61654333-ca77-4a1e-822f-8a10d1b3ad73\" (UID: \"61654333-ca77-4a1e-822f-8a10d1b3ad73\") "
Jul 16 21:22:41.688966 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.688941    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-bundle" (OuterVolumeSpecName: "bundle") pod "61654333-ca77-4a1e-822f-8a10d1b3ad73" (UID: "61654333-ca77-4a1e-822f-8a10d1b3ad73"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:41.690823 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.690800    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61654333-ca77-4a1e-822f-8a10d1b3ad73-kube-api-access-gxq2p" (OuterVolumeSpecName: "kube-api-access-gxq2p") pod "61654333-ca77-4a1e-822f-8a10d1b3ad73" (UID: "61654333-ca77-4a1e-822f-8a10d1b3ad73"). InnerVolumeSpecName "kube-api-access-gxq2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:22:41.693412 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.693386    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-util" (OuterVolumeSpecName: "util") pod "61654333-ca77-4a1e-822f-8a10d1b3ad73" (UID: "61654333-ca77-4a1e-822f-8a10d1b3ad73"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:41.790005 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.789966    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxq2p\" (UniqueName: \"kubernetes.io/projected/61654333-ca77-4a1e-822f-8a10d1b3ad73-kube-api-access-gxq2p\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:41.790005 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.789996    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-util\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:41.790005 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:41.790006    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61654333-ca77-4a1e-822f-8a10d1b3ad73-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.374382 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.374324    2572 generic.go:356] "Generic (PLEG): container finished" podID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerID="7a08b1098403725811c839b3857e935912a46b885a970d119cf726d526a10b66" exitCode=0
Jul 16 21:22:42.374768 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.374412    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj" event={"ID":"fe78c421-68ae-4b06-983c-f3ba2bd24747","Type":"ContainerDied","Data":"7a08b1098403725811c839b3857e935912a46b885a970d119cf726d526a10b66"}
Jul 16 21:22:42.376323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.376303    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h"
Jul 16 21:22:42.376467 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.376333    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h" event={"ID":"61654333-ca77-4a1e-822f-8a10d1b3ad73","Type":"ContainerDied","Data":"9b403aefffd2d0cda8d6365b9344e9797674e488cbdb143e2a2bce6ed8b1259b"}
Jul 16 21:22:42.376467 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.376391    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b403aefffd2d0cda8d6365b9344e9797674e488cbdb143e2a2bce6ed8b1259b"
Jul 16 21:22:42.544479 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.544451    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:42.547391 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.547371    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:42.701780 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.701689    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-bundle\") pod \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") "
Jul 16 21:22:42.701780 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.701777    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-util\") pod \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") "
Jul 16 21:22:42.702017 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.701805    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-bundle\") pod \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") "
Jul 16 21:22:42.702017 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.701840    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-util\") pod \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") "
Jul 16 21:22:42.702017 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.701885    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chp9j\" (UniqueName: \"kubernetes.io/projected/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-kube-api-access-chp9j\") pod \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\" (UID: \"bd963ca4-8972-4a62-840c-d4cbdaa88cf7\") "
Jul 16 21:22:42.702017 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.701909    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmlxs\" (UniqueName: \"kubernetes.io/projected/b6d8de27-f4d2-48c6-8e21-28883d5025cd-kube-api-access-jmlxs\") pod \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\" (UID: \"b6d8de27-f4d2-48c6-8e21-28883d5025cd\") "
Jul 16 21:22:42.702393 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.702338    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-bundle" (OuterVolumeSpecName: "bundle") pod "b6d8de27-f4d2-48c6-8e21-28883d5025cd" (UID: "b6d8de27-f4d2-48c6-8e21-28883d5025cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.702513 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.702490    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-bundle" (OuterVolumeSpecName: "bundle") pod "bd963ca4-8972-4a62-840c-d4cbdaa88cf7" (UID: "bd963ca4-8972-4a62-840c-d4cbdaa88cf7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.704155 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.704129    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-kube-api-access-chp9j" (OuterVolumeSpecName: "kube-api-access-chp9j") pod "bd963ca4-8972-4a62-840c-d4cbdaa88cf7" (UID: "bd963ca4-8972-4a62-840c-d4cbdaa88cf7"). InnerVolumeSpecName "kube-api-access-chp9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:22:42.704443 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.704421    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d8de27-f4d2-48c6-8e21-28883d5025cd-kube-api-access-jmlxs" (OuterVolumeSpecName: "kube-api-access-jmlxs") pod "b6d8de27-f4d2-48c6-8e21-28883d5025cd" (UID: "b6d8de27-f4d2-48c6-8e21-28883d5025cd"). InnerVolumeSpecName "kube-api-access-jmlxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:22:42.708675 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.708640    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-util" (OuterVolumeSpecName: "util") pod "b6d8de27-f4d2-48c6-8e21-28883d5025cd" (UID: "b6d8de27-f4d2-48c6-8e21-28883d5025cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.709948 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.709923    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-util" (OuterVolumeSpecName: "util") pod "bd963ca4-8972-4a62-840c-d4cbdaa88cf7" (UID: "bd963ca4-8972-4a62-840c-d4cbdaa88cf7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.803445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.803397    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-util\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.803445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.803436    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chp9j\" (UniqueName: \"kubernetes.io/projected/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-kube-api-access-chp9j\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.803445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.803451    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmlxs\" (UniqueName: \"kubernetes.io/projected/b6d8de27-f4d2-48c6-8e21-28883d5025cd-kube-api-access-jmlxs\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.803679 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.803464    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.803679 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.803475    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bd963ca4-8972-4a62-840c-d4cbdaa88cf7-util\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.803679 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:42.803484    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6d8de27-f4d2-48c6-8e21-28883d5025cd-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:43.382874 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.382846    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8"
Jul 16 21:22:43.383290 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.382881    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8" event={"ID":"b6d8de27-f4d2-48c6-8e21-28883d5025cd","Type":"ContainerDied","Data":"d7a33366a3376a59fb02136fc22f4fa8affeda86ef8ff54ad30930fe829d2ee7"}
Jul 16 21:22:43.383290 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.382958    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7a33366a3376a59fb02136fc22f4fa8affeda86ef8ff54ad30930fe829d2ee7"
Jul 16 21:22:43.385023 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.384995    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw" event={"ID":"bd963ca4-8972-4a62-840c-d4cbdaa88cf7","Type":"ContainerDied","Data":"5185c63e293d3541798210e6e5c97e06b57de6080ed26fa83587c1856462d550"}
Jul 16 21:22:43.385132 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.385053    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5185c63e293d3541798210e6e5c97e06b57de6080ed26fa83587c1856462d550"
Jul 16 21:22:43.385205 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.385178    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw"
Jul 16 21:22:43.507980 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.507951    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:43.609583 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.609544    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-util\") pod \"fe78c421-68ae-4b06-983c-f3ba2bd24747\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") "
Jul 16 21:22:43.609790 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.609610    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-bundle\") pod \"fe78c421-68ae-4b06-983c-f3ba2bd24747\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") "
Jul 16 21:22:43.609790 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.609636    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfpgn\" (UniqueName: \"kubernetes.io/projected/fe78c421-68ae-4b06-983c-f3ba2bd24747-kube-api-access-xfpgn\") pod \"fe78c421-68ae-4b06-983c-f3ba2bd24747\" (UID: \"fe78c421-68ae-4b06-983c-f3ba2bd24747\") "
Jul 16 21:22:43.610153 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.610127    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-bundle" (OuterVolumeSpecName: "bundle") pod "fe78c421-68ae-4b06-983c-f3ba2bd24747" (UID: "fe78c421-68ae-4b06-983c-f3ba2bd24747"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:43.611873 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.611844    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe78c421-68ae-4b06-983c-f3ba2bd24747-kube-api-access-xfpgn" (OuterVolumeSpecName: "kube-api-access-xfpgn") pod "fe78c421-68ae-4b06-983c-f3ba2bd24747" (UID: "fe78c421-68ae-4b06-983c-f3ba2bd24747"). InnerVolumeSpecName "kube-api-access-xfpgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:22:43.615047 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.615023    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-util" (OuterVolumeSpecName: "util") pod "fe78c421-68ae-4b06-983c-f3ba2bd24747" (UID: "fe78c421-68ae-4b06-983c-f3ba2bd24747"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:43.710853 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.710819    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:43.710853 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.710849    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfpgn\" (UniqueName: \"kubernetes.io/projected/fe78c421-68ae-4b06-983c-f3ba2bd24747-kube-api-access-xfpgn\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:43.710853 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:43.710861    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe78c421-68ae-4b06-983c-f3ba2bd24747-util\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:22:44.391845 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:44.391809    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj" event={"ID":"fe78c421-68ae-4b06-983c-f3ba2bd24747","Type":"ContainerDied","Data":"145cb57aa36892b18925d53832fd6eb0997c029c10e96b81c8500fb94f862be0"}
Jul 16 21:22:44.392223 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:44.391849    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj"
Jul 16 21:22:44.392223 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:44.391859    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="145cb57aa36892b18925d53832fd6eb0997c029c10e96b81c8500fb94f862be0"
Jul 16 21:22:56.737687 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737606    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"]
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737944    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerName="util"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737954    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerName="util"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737962    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerName="pull"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737967    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerName="pull"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737974    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerName="util"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737980    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerName="util"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737988    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerName="pull"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.737993    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerName="pull"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738002    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerName="util"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738007    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerName="util"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738019    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerName="extract"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738024    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerName="extract"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738029    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerName="pull"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738034    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerName="pull"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738039    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerName="extract"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738044    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerName="extract"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738049    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerName="extract"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738054    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerName="extract"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738060    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerName="extract"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738065    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerName="extract"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738075    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerName="util"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738080    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerName="util"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738087    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerName="pull"
Jul 16 21:22:56.738127 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738091    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerName="pull"
Jul 16 21:22:56.738848 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738143    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd963ca4-8972-4a62-840c-d4cbdaa88cf7" containerName="extract"
Jul 16 21:22:56.738848 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738152    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="61654333-ca77-4a1e-822f-8a10d1b3ad73" containerName="extract"
Jul 16 21:22:56.738848 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738160    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6d8de27-f4d2-48c6-8e21-28883d5025cd" containerName="extract"
Jul 16 21:22:56.738848 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.738166    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe78c421-68ae-4b06-983c-f3ba2bd24747" containerName="extract"
Jul 16 21:22:56.740117 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.740100    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"
Jul 16 21:22:56.742822 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.742794    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 21:22:56.742962 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.742867    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 21:22:56.742962 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.742892    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-5qrtk\""
Jul 16 21:22:56.752411 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.752370    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"]
Jul 16 21:22:56.829041 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.829002    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk9b4\" (UniqueName: \"kubernetes.io/projected/38bd0e8d-85bf-489f-96e8-1b4685a8ebe7-kube-api-access-sk9b4\") pod \"authorino-operator-75cc8d5cb7-4bgf5\" (UID: \"38bd0e8d-85bf-489f-96e8-1b4685a8ebe7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"
Jul 16 21:22:56.929461 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.929424    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk9b4\" (UniqueName: \"kubernetes.io/projected/38bd0e8d-85bf-489f-96e8-1b4685a8ebe7-kube-api-access-sk9b4\") pod \"authorino-operator-75cc8d5cb7-4bgf5\" (UID: \"38bd0e8d-85bf-489f-96e8-1b4685a8ebe7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"
Jul 16 21:22:56.941920 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:56.941882    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk9b4\" (UniqueName: \"kubernetes.io/projected/38bd0e8d-85bf-489f-96e8-1b4685a8ebe7-kube-api-access-sk9b4\") pod \"authorino-operator-75cc8d5cb7-4bgf5\" (UID: \"38bd0e8d-85bf-489f-96e8-1b4685a8ebe7\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"
Jul 16 21:22:57.050456 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:57.050420    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"
Jul 16 21:22:57.208142 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:57.208002    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"]
Jul 16 21:22:57.210980 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:22:57.210949    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bd0e8d_85bf_489f_96e8_1b4685a8ebe7.slice/crio-6baebc80cce3b636ff8c91d83471abbdb5f21802bb69db1342176f57ac6b75dc WatchSource:0}: Error finding container 6baebc80cce3b636ff8c91d83471abbdb5f21802bb69db1342176f57ac6b75dc: Status 404 returned error can't find the container with id 6baebc80cce3b636ff8c91d83471abbdb5f21802bb69db1342176f57ac6b75dc
Jul 16 21:22:57.458007 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:57.457916    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5" event={"ID":"38bd0e8d-85bf-489f-96e8-1b4685a8ebe7","Type":"ContainerStarted","Data":"6baebc80cce3b636ff8c91d83471abbdb5f21802bb69db1342176f57ac6b75dc"}
Jul 16 21:22:59.471148 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:59.471116    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5" event={"ID":"38bd0e8d-85bf-489f-96e8-1b4685a8ebe7","Type":"ContainerStarted","Data":"6454f3f6803430eab72d80e7d1af902bf3f2de400d05dbaf846ea56db92b3463"}
Jul 16 21:22:59.471554 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:59.471218    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"
Jul 16 21:22:59.489829 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:22:59.489781    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5" podStartSLOduration=1.327912157 podStartE2EDuration="3.489762702s" totalImagesPullingTime="2.161850545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:22:59.487448344 +0000 UTC m=+444.998608079" watchObservedRunningTime="2026-07-16 21:22:59.489762702 +0000 UTC m=+445.000922439"
Jul 16 21:23:10.479921 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:10.479888    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4bgf5"
Jul 16 21:23:16.372967 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:16.372929    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65f7b757f5-mm2tr"]
Jul 16 21:23:26.260689 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.260651    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"]
Jul 16 21:23:26.268484 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.268454    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.271735 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.271706    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zxb6j\""
Jul 16 21:23:26.278491 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.278465    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"]
Jul 16 21:23:26.419722 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.419683    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rcffn\" (UID: \"e603a9bf-7b63-4cf5-a3b2-df91e337acdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.419896 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.419741    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmtn\" (UniqueName: \"kubernetes.io/projected/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-kube-api-access-cbmtn\") pod \"kuadrant-operator-controller-manager-55467d9f7-rcffn\" (UID: \"e603a9bf-7b63-4cf5-a3b2-df91e337acdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.516632 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.516552    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"]
Jul 16 21:23:26.516854 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:23:26.516834    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-cbmtn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn" podUID="e603a9bf-7b63-4cf5-a3b2-df91e337acdd"
Jul 16 21:23:26.521295 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.521255    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rcffn\" (UID: \"e603a9bf-7b63-4cf5-a3b2-df91e337acdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.521472 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.521405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmtn\" (UniqueName: \"kubernetes.io/projected/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-kube-api-access-cbmtn\") pod \"kuadrant-operator-controller-manager-55467d9f7-rcffn\" (UID: \"e603a9bf-7b63-4cf5-a3b2-df91e337acdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.521739 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.521718    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rcffn\" (UID: \"e603a9bf-7b63-4cf5-a3b2-df91e337acdd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.523585 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:23:26.523563    2572 projected.go:196] Error preparing data for projected volume kube-api-access-cbmtn for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn: failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-133-67.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object
Jul 16 21:23:26.523707 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:23:26.523648    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-kube-api-access-cbmtn podName:e603a9bf-7b63-4cf5-a3b2-df91e337acdd nodeName:}" failed. No retries permitted until 2026-07-16 21:23:27.023626222 +0000 UTC m=+472.534785933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cbmtn" (UniqueName: "kubernetes.io/projected/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-kube-api-access-cbmtn") pod "kuadrant-operator-controller-manager-55467d9f7-rcffn" (UID: "e603a9bf-7b63-4cf5-a3b2-df91e337acdd") : failed to fetch token: serviceaccounts "kuadrant-operator-controller-manager" is forbidden: User "system:node:ip-10-0-133-67.ec2.internal" cannot create resource "serviceaccounts/token" in API group "" in the namespace "kuadrant-system": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object
Jul 16 21:23:26.523833 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.523807    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"]
Jul 16 21:23:26.544537 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.544499    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"]
Jul 16 21:23:26.548627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.548604    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:26.565185 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.565152    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"]
Jul 16 21:23:26.611153 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.611121    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.613658 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:23:26.613622    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rcffn\" is forbidden: User \"system:node:ip-10-0-133-67.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object" podUID="e603a9bf-7b63-4cf5-a3b2-df91e337acdd" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.617577 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.617554    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.620009 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:23:26.619977    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rcffn\" is forbidden: User \"system:node:ip-10-0-133-67.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object" podUID="e603a9bf-7b63-4cf5-a3b2-df91e337acdd" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:26.723077 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.723020    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-extensions-socket-volume\") pod \"e603a9bf-7b63-4cf5-a3b2-df91e337acdd\" (UID: \"e603a9bf-7b63-4cf5-a3b2-df91e337acdd\") "
Jul 16 21:23:26.723284 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.723136    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbbh9\" (UniqueName: \"kubernetes.io/projected/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-kube-api-access-zbbh9\") pod \"kuadrant-operator-controller-manager-55467d9f7-pz4vn\" (UID: \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:26.723284 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.723190    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pz4vn\" (UID: \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:26.723436 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.723282    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e603a9bf-7b63-4cf5-a3b2-df91e337acdd" (UID: "e603a9bf-7b63-4cf5-a3b2-df91e337acdd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:23:26.723436 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.723382    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmtn\" (UniqueName: \"kubernetes.io/projected/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-kube-api-access-cbmtn\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:26.824486 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.824441    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbbh9\" (UniqueName: \"kubernetes.io/projected/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-kube-api-access-zbbh9\") pod \"kuadrant-operator-controller-manager-55467d9f7-pz4vn\" (UID: \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:26.824691 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.824534    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pz4vn\" (UID: \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:26.824691 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.824623    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e603a9bf-7b63-4cf5-a3b2-df91e337acdd-extensions-socket-volume\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:26.824927 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.824907    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pz4vn\" (UID: \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:26.843720 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.843688    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbbh9\" (UniqueName: \"kubernetes.io/projected/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-kube-api-access-zbbh9\") pod \"kuadrant-operator-controller-manager-55467d9f7-pz4vn\" (UID: \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:26.859566 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:26.859529    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:27.010508 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:27.010477    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"]
Jul 16 21:23:27.012813 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:23:27.012784    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1add1136_d2bc_4c82_a64a_5a1ea748fe5f.slice/crio-f73dfb63b7fcf6424a92d4695478f2efed6e776063e9ad8884ac3cad8c66e3db WatchSource:0}: Error finding container f73dfb63b7fcf6424a92d4695478f2efed6e776063e9ad8884ac3cad8c66e3db: Status 404 returned error can't find the container with id f73dfb63b7fcf6424a92d4695478f2efed6e776063e9ad8884ac3cad8c66e3db
Jul 16 21:23:27.249249 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:27.249172    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e603a9bf-7b63-4cf5-a3b2-df91e337acdd" path="/var/lib/kubelet/pods/e603a9bf-7b63-4cf5-a3b2-df91e337acdd/volumes"
Jul 16 21:23:27.617689 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:27.617647    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn" event={"ID":"1add1136-d2bc-4c82-a64a-5a1ea748fe5f","Type":"ContainerStarted","Data":"f73dfb63b7fcf6424a92d4695478f2efed6e776063e9ad8884ac3cad8c66e3db"}
Jul 16 21:23:27.618133 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:27.617659    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:27.622010 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:23:27.621977    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-rcffn\" is forbidden: User \"system:node:ip-10-0-133-67.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-67.ec2.internal' and this object" podUID="e603a9bf-7b63-4cf5-a3b2-df91e337acdd" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rcffn"
Jul 16 21:23:32.647549 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:32.647505    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn" event={"ID":"1add1136-d2bc-4c82-a64a-5a1ea748fe5f","Type":"ContainerStarted","Data":"821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c"}
Jul 16 21:23:32.647962 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:32.647572    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:32.672983 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:32.672927    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn" podStartSLOduration=1.704718551 podStartE2EDuration="6.672912083s" totalImagesPullingTime="4.968193532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:23:32.669216112 +0000 UTC m=+478.180375847" watchObservedRunningTime="2026-07-16 21:23:32.672912083 +0000 UTC m=+478.184071816"
Jul 16 21:23:41.396073 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.396027    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65f7b757f5-mm2tr" podUID="e24eaf0a-81cc-48d3-8e00-e1b7a9617868" containerName="console" containerID="cri-o://88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7" gracePeriod=15
Jul 16 21:23:41.647464 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.647397    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:23:41.699983 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.699948    2572 generic.go:356] "Generic (PLEG): container finished" podID="e24eaf0a-81cc-48d3-8e00-e1b7a9617868" containerID="88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7" exitCode=0
Jul 16 21:23:41.700174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.700003    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65f7b757f5-mm2tr"
Jul 16 21:23:41.700174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.700031    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f7b757f5-mm2tr" event={"ID":"e24eaf0a-81cc-48d3-8e00-e1b7a9617868","Type":"ContainerDied","Data":"88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7"}
Jul 16 21:23:41.700174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.700083    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65f7b757f5-mm2tr" event={"ID":"e24eaf0a-81cc-48d3-8e00-e1b7a9617868","Type":"ContainerDied","Data":"e54915b26fe8d083041ecaae8bb845593dba2d0129ccf28cce7edc9f5429c40d"}
Jul 16 21:23:41.700174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.700105    2572 scope.go:117] "RemoveContainer" containerID="88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7"
Jul 16 21:23:41.711614 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.711590    2572 scope.go:117] "RemoveContainer" containerID="88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7"
Jul 16 21:23:41.712004 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:23:41.711983    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7\": container with ID starting with 88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7 not found: ID does not exist" containerID="88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7"
Jul 16 21:23:41.712105 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.712012    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7"} err="failed to get container status \"88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7\": rpc error: code = NotFound desc = could not find container \"88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7\": container with ID starting with 88fe7e1d1073f6f901727a0469ea4bfce7032e78d32db9089a3d9321312ccef7 not found: ID does not exist"
Jul 16 21:23:41.748528 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.748491    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-serving-cert\") pod \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") "
Jul 16 21:23:41.748528 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.748528    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-service-ca\") pod \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") "
Jul 16 21:23:41.748779 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.748581    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvt2l\" (UniqueName: \"kubernetes.io/projected/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-kube-api-access-vvt2l\") pod \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") "
Jul 16 21:23:41.748779 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.748642    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-trusted-ca-bundle\") pod \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") "
Jul 16 21:23:41.748779 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.748668    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-config\") pod \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") "
Jul 16 21:23:41.748779 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.748717    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-oauth-serving-cert\") pod \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") "
Jul 16 21:23:41.748779 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.748742    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-oauth-config\") pod \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\" (UID: \"e24eaf0a-81cc-48d3-8e00-e1b7a9617868\") "
Jul 16 21:23:41.749103 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.749072    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-service-ca" (OuterVolumeSpecName: "service-ca") pod "e24eaf0a-81cc-48d3-8e00-e1b7a9617868" (UID: "e24eaf0a-81cc-48d3-8e00-e1b7a9617868"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:23:41.749103 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.749080    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-config" (OuterVolumeSpecName: "console-config") pod "e24eaf0a-81cc-48d3-8e00-e1b7a9617868" (UID: "e24eaf0a-81cc-48d3-8e00-e1b7a9617868"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:23:41.749103 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.749099    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e24eaf0a-81cc-48d3-8e00-e1b7a9617868" (UID: "e24eaf0a-81cc-48d3-8e00-e1b7a9617868"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:23:41.749258 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.749145    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e24eaf0a-81cc-48d3-8e00-e1b7a9617868" (UID: "e24eaf0a-81cc-48d3-8e00-e1b7a9617868"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:23:41.750855 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.750830    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e24eaf0a-81cc-48d3-8e00-e1b7a9617868" (UID: "e24eaf0a-81cc-48d3-8e00-e1b7a9617868"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:23:41.750961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.750855    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e24eaf0a-81cc-48d3-8e00-e1b7a9617868" (UID: "e24eaf0a-81cc-48d3-8e00-e1b7a9617868"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:23:41.750961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.750867    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-kube-api-access-vvt2l" (OuterVolumeSpecName: "kube-api-access-vvt2l") pod "e24eaf0a-81cc-48d3-8e00-e1b7a9617868" (UID: "e24eaf0a-81cc-48d3-8e00-e1b7a9617868"). InnerVolumeSpecName "kube-api-access-vvt2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:23:41.849537 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.849498    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:41.849537 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.849530    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvt2l\" (UniqueName: \"kubernetes.io/projected/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-kube-api-access-vvt2l\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:41.849537 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.849542    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-trusted-ca-bundle\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:41.849537 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.849551    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-config\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:41.849836 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.849560    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-oauth-serving-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:41.849836 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.849568    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-oauth-config\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:41.849836 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:41.849576    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e24eaf0a-81cc-48d3-8e00-e1b7a9617868-console-serving-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:42.022707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:42.022667    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65f7b757f5-mm2tr"]
Jul 16 21:23:42.026938 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:42.026907    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65f7b757f5-mm2tr"]
Jul 16 21:23:43.254072 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:43.254038    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e24eaf0a-81cc-48d3-8e00-e1b7a9617868" path="/var/lib/kubelet/pods/e24eaf0a-81cc-48d3-8e00-e1b7a9617868/volumes"
Jul 16 21:23:43.655735 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:43.655702    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:56.000645 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.000607    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"]
Jul 16 21:23:56.001098 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.000794    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn" podUID="1add1136-d2bc-4c82-a64a-5a1ea748fe5f" containerName="manager" containerID="cri-o://821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c" gracePeriod=10
Jul 16 21:23:56.262851 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.262789    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:56.377823 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.377789    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbbh9\" (UniqueName: \"kubernetes.io/projected/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-kube-api-access-zbbh9\") pod \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\" (UID: \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\") "
Jul 16 21:23:56.377992 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.377845    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-extensions-socket-volume\") pod \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\" (UID: \"1add1136-d2bc-4c82-a64a-5a1ea748fe5f\") "
Jul 16 21:23:56.378289 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.378255    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1add1136-d2bc-4c82-a64a-5a1ea748fe5f" (UID: "1add1136-d2bc-4c82-a64a-5a1ea748fe5f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:23:56.379937 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.379911    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-kube-api-access-zbbh9" (OuterVolumeSpecName: "kube-api-access-zbbh9") pod "1add1136-d2bc-4c82-a64a-5a1ea748fe5f" (UID: "1add1136-d2bc-4c82-a64a-5a1ea748fe5f"). InnerVolumeSpecName "kube-api-access-zbbh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:23:56.479330 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.479298    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-extensions-socket-volume\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:56.479330 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.479327    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbbh9\" (UniqueName: \"kubernetes.io/projected/1add1136-d2bc-4c82-a64a-5a1ea748fe5f-kube-api-access-zbbh9\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:23:56.782800 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.782766    2572 generic.go:356] "Generic (PLEG): container finished" podID="1add1136-d2bc-4c82-a64a-5a1ea748fe5f" containerID="821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c" exitCode=0
Jul 16 21:23:56.782971 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.782815    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"
Jul 16 21:23:56.782971 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.782851    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn" event={"ID":"1add1136-d2bc-4c82-a64a-5a1ea748fe5f","Type":"ContainerDied","Data":"821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c"}
Jul 16 21:23:56.782971 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.782927    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn" event={"ID":"1add1136-d2bc-4c82-a64a-5a1ea748fe5f","Type":"ContainerDied","Data":"f73dfb63b7fcf6424a92d4695478f2efed6e776063e9ad8884ac3cad8c66e3db"}
Jul 16 21:23:56.782971 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.782953    2572 scope.go:117] "RemoveContainer" containerID="821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c"
Jul 16 21:23:56.792806 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.792788    2572 scope.go:117] "RemoveContainer" containerID="821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c"
Jul 16 21:23:56.793096 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:23:56.793080    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c\": container with ID starting with 821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c not found: ID does not exist" containerID="821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c"
Jul 16 21:23:56.793130 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.793106    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c"} err="failed to get container status \"821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c\": rpc error: code = NotFound desc = could not find container \"821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c\": container with ID starting with 821ef2b90199efb8f85563d09ef66b4afbcf10ca36f35ea7360ecee33cfb423c not found: ID does not exist"
Jul 16 21:23:56.808623 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.808587    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"]
Jul 16 21:23:56.815034 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:56.815006    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pz4vn"]
Jul 16 21:23:57.250195 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:23:57.250117    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1add1136-d2bc-4c82-a64a-5a1ea748fe5f" path="/var/lib/kubelet/pods/1add1136-d2bc-4c82-a64a-5a1ea748fe5f/volumes"
Jul 16 21:24:00.009909 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.009874    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"]
Jul 16 21:24:00.010557 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.010531    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1add1136-d2bc-4c82-a64a-5a1ea748fe5f" containerName="manager"
Jul 16 21:24:00.010714 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.010695    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1add1136-d2bc-4c82-a64a-5a1ea748fe5f" containerName="manager"
Jul 16 21:24:00.010832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.010736    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e24eaf0a-81cc-48d3-8e00-e1b7a9617868" containerName="console"
Jul 16 21:24:00.010832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.010743    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e24eaf0a-81cc-48d3-8e00-e1b7a9617868" containerName="console"
Jul 16 21:24:00.010924 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.010856    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1add1136-d2bc-4c82-a64a-5a1ea748fe5f" containerName="manager"
Jul 16 21:24:00.010924 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.010869    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e24eaf0a-81cc-48d3-8e00-e1b7a9617868" containerName="console"
Jul 16 21:24:00.015286 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.015264    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.017729 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.017707    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nt95g\""
Jul 16 21:24:00.027867 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.027839    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"]
Jul 16 21:24:00.114303 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114267    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.114303 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114308    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.114562 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114327    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa0262c-6293-4621-81bb-fe494452904d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.114562 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114408    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa0262c-6293-4621-81bb-fe494452904d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.114562 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114454    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa0262c-6293-4621-81bb-fe494452904d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.114562 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114476    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.114562 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114517    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.114562 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114547    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.114769 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.114575    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmg96\" (UniqueName: \"kubernetes.io/projected/cfa0262c-6293-4621-81bb-fe494452904d-kube-api-access-hmg96\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.215662 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215615    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa0262c-6293-4621-81bb-fe494452904d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.215662 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215668    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa0262c-6293-4621-81bb-fe494452904d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.215920 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215722    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa0262c-6293-4621-81bb-fe494452904d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.215920 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215751    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.215920 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215795    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.215920 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215830    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.215920 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215869    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmg96\" (UniqueName: \"kubernetes.io/projected/cfa0262c-6293-4621-81bb-fe494452904d-kube-api-access-hmg96\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.215920 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215902    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.216195 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.215941    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.216260 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.216193    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.216319 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.216289    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.216424 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.216401    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.216491 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.216470    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cfa0262c-6293-4621-81bb-fe494452904d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.216491 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.216476    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.218179 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.218155    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cfa0262c-6293-4621-81bb-fe494452904d-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.218278 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.218195    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cfa0262c-6293-4621-81bb-fe494452904d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.227894 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.227864    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmg96\" (UniqueName: \"kubernetes.io/projected/cfa0262c-6293-4621-81bb-fe494452904d-kube-api-access-hmg96\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.234973 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.234948    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cfa0262c-6293-4621-81bb-fe494452904d-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xbb86\" (UID: \"cfa0262c-6293-4621-81bb-fe494452904d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.325947 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.325909    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:00.466499 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.466471    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"]
Jul 16 21:24:00.467913 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:24:00.467888    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfa0262c_6293_4621_81bb_fe494452904d.slice/crio-16080c250429eebd275bb27f67493962ba435a74e52bc9f57cd6495fb17bdf00 WatchSource:0}: Error finding container 16080c250429eebd275bb27f67493962ba435a74e52bc9f57cd6495fb17bdf00: Status 404 returned error can't find the container with id 16080c250429eebd275bb27f67493962ba435a74e52bc9f57cd6495fb17bdf00
Jul 16 21:24:00.470005 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.469975    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:24:00.470063 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.470038    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:24:00.470095 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.470067    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:24:00.808434 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.808392    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" event={"ID":"cfa0262c-6293-4621-81bb-fe494452904d","Type":"ContainerStarted","Data":"1d36eb6b2098473bdc2308c777a83537be342eec70466bbbe2c82f3b6d705c8d"}
Jul 16 21:24:00.808622 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.808456    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" event={"ID":"cfa0262c-6293-4621-81bb-fe494452904d","Type":"ContainerStarted","Data":"16080c250429eebd275bb27f67493962ba435a74e52bc9f57cd6495fb17bdf00"}
Jul 16 21:24:00.832546 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:00.832490    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" podStartSLOduration=1.832471855 podStartE2EDuration="1.832471855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:23:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:24:00.829733704 +0000 UTC m=+506.340893452" watchObservedRunningTime="2026-07-16 21:24:00.832471855 +0000 UTC m=+506.343631589"
Jul 16 21:24:01.326684 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:01.326650    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:01.331972 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:01.331946    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:01.814419 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:01.814390    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:01.815340 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:01.815324    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:24:38.847275 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.847234    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"]
Jul 16 21:24:38.852195 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.852171    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:38.854748 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.854728    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:24:38.855172 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.855157    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 16 21:24:38.855521 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.855504    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 16 21:24:38.856019 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.856003    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qf9bs\""
Jul 16 21:24:38.868998 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.868967    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"]
Jul 16 21:24:38.953429 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.953387    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcrrd\" (UniqueName: \"kubernetes.io/projected/28c36354-901b-4cb9-ad5a-5ce27c091652-kube-api-access-bcrrd\") pod \"postgres-69fb4594cf-nwdcj\" (UID: \"28c36354-901b-4cb9-ad5a-5ce27c091652\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:38.953625 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:38.953456    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28c36354-901b-4cb9-ad5a-5ce27c091652-data\") pod \"postgres-69fb4594cf-nwdcj\" (UID: \"28c36354-901b-4cb9-ad5a-5ce27c091652\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:39.054632 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:39.054592    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcrrd\" (UniqueName: \"kubernetes.io/projected/28c36354-901b-4cb9-ad5a-5ce27c091652-kube-api-access-bcrrd\") pod \"postgres-69fb4594cf-nwdcj\" (UID: \"28c36354-901b-4cb9-ad5a-5ce27c091652\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:39.054806 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:39.054648    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28c36354-901b-4cb9-ad5a-5ce27c091652-data\") pod \"postgres-69fb4594cf-nwdcj\" (UID: \"28c36354-901b-4cb9-ad5a-5ce27c091652\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:39.054976 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:39.054962    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28c36354-901b-4cb9-ad5a-5ce27c091652-data\") pod \"postgres-69fb4594cf-nwdcj\" (UID: \"28c36354-901b-4cb9-ad5a-5ce27c091652\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:39.065043 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:39.065014    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcrrd\" (UniqueName: \"kubernetes.io/projected/28c36354-901b-4cb9-ad5a-5ce27c091652-kube-api-access-bcrrd\") pod \"postgres-69fb4594cf-nwdcj\" (UID: \"28c36354-901b-4cb9-ad5a-5ce27c091652\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:39.169331 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:39.169240    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:39.316589 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:39.316556    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"]
Jul 16 21:24:39.319819 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:24:39.319794    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c36354_901b_4cb9_ad5a_5ce27c091652.slice/crio-cd37cf1357369dd6edf2e72674dccea9aa5ab1cf6fe2abc588617804e821ee5d WatchSource:0}: Error finding container cd37cf1357369dd6edf2e72674dccea9aa5ab1cf6fe2abc588617804e821ee5d: Status 404 returned error can't find the container with id cd37cf1357369dd6edf2e72674dccea9aa5ab1cf6fe2abc588617804e821ee5d
Jul 16 21:24:40.011079 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:40.011042    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj" event={"ID":"28c36354-901b-4cb9-ad5a-5ce27c091652","Type":"ContainerStarted","Data":"cd37cf1357369dd6edf2e72674dccea9aa5ab1cf6fe2abc588617804e821ee5d"}
Jul 16 21:24:45.044624 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:45.044582    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj" event={"ID":"28c36354-901b-4cb9-ad5a-5ce27c091652","Type":"ContainerStarted","Data":"a2634c956dfdf848c475df781881cf122d31874e3fbbc432801bb6b0fc9fad17"}
Jul 16 21:24:45.045036 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:45.044652    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:45.063843 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:45.063782    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj" podStartSLOduration=1.812587494 podStartE2EDuration="7.06376631s" totalImagesPullingTime="5.251178816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:24:45.062587106 +0000 UTC m=+550.573746841" watchObservedRunningTime="2026-07-16 21:24:45.06376631 +0000 UTC m=+550.574926043"
Jul 16 21:24:51.080374 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:51.080305    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nwdcj"
Jul 16 21:24:58.669253 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.669218    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5659cfddb-6jgv6"]
Jul 16 21:24:58.672789 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.672765    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:24:58.676342 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.676319    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 21:24:58.676464 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.676372    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 21:24:58.676613 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.676588    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mqzw5\""
Jul 16 21:24:58.684201 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.684174    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5659cfddb-6jgv6"]
Jul 16 21:24:58.733789 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.733752    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwglq\" (UniqueName: \"kubernetes.io/projected/5e214903-ae68-44f4-a559-3e93f69a2cc7-kube-api-access-mwglq\") pod \"maas-controller-5659cfddb-6jgv6\" (UID: \"5e214903-ae68-44f4-a559-3e93f69a2cc7\") " pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:24:58.733980 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.733806    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e214903-ae68-44f4-a559-3e93f69a2cc7-webhook-cert\") pod \"maas-controller-5659cfddb-6jgv6\" (UID: \"5e214903-ae68-44f4-a559-3e93f69a2cc7\") " pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:24:58.834748 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.834716    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwglq\" (UniqueName: \"kubernetes.io/projected/5e214903-ae68-44f4-a559-3e93f69a2cc7-kube-api-access-mwglq\") pod \"maas-controller-5659cfddb-6jgv6\" (UID: \"5e214903-ae68-44f4-a559-3e93f69a2cc7\") " pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:24:58.834958 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.834768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e214903-ae68-44f4-a559-3e93f69a2cc7-webhook-cert\") pod \"maas-controller-5659cfddb-6jgv6\" (UID: \"5e214903-ae68-44f4-a559-3e93f69a2cc7\") " pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:24:58.837224 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.837197    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e214903-ae68-44f4-a559-3e93f69a2cc7-webhook-cert\") pod \"maas-controller-5659cfddb-6jgv6\" (UID: \"5e214903-ae68-44f4-a559-3e93f69a2cc7\") " pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:24:58.844478 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.844450    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwglq\" (UniqueName: \"kubernetes.io/projected/5e214903-ae68-44f4-a559-3e93f69a2cc7-kube-api-access-mwglq\") pod \"maas-controller-5659cfddb-6jgv6\" (UID: \"5e214903-ae68-44f4-a559-3e93f69a2cc7\") " pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:24:58.983320 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:58.983234    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:24:59.123236 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:24:59.123210    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5659cfddb-6jgv6"]
Jul 16 21:24:59.124885 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:24:59.124856    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e214903_ae68_44f4_a559_3e93f69a2cc7.slice/crio-668ce83fe46263e80c0493743f1971a4523508340b6e8d297853bcd5d1643f68 WatchSource:0}: Error finding container 668ce83fe46263e80c0493743f1971a4523508340b6e8d297853bcd5d1643f68: Status 404 returned error can't find the container with id 668ce83fe46263e80c0493743f1971a4523508340b6e8d297853bcd5d1643f68
Jul 16 21:25:00.129774 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:00.129731    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-6jgv6" event={"ID":"5e214903-ae68-44f4-a559-3e93f69a2cc7","Type":"ContainerStarted","Data":"668ce83fe46263e80c0493743f1971a4523508340b6e8d297853bcd5d1643f68"}
Jul 16 21:25:02.143432 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:02.143388    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-6jgv6" event={"ID":"5e214903-ae68-44f4-a559-3e93f69a2cc7","Type":"ContainerStarted","Data":"203aa6a548eda81f8c4dc68c8208d375ce1429d2ea54e0fc62963d10b5274f69"}
Jul 16 21:25:02.143879 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:02.143672    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:25:02.169343 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:02.169286    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5659cfddb-6jgv6" podStartSLOduration=1.7995221369999999 podStartE2EDuration="4.169268225s" totalImagesPullingTime="2.369746088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:24:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:25:02.168596898 +0000 UTC m=+567.679756632" watchObservedRunningTime="2026-07-16 21:25:02.169268225 +0000 UTC m=+567.680427958"
Jul 16 21:25:13.151327 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:13.151290    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:25:42.354714 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:42.354679    2572 generic.go:356] "Generic (PLEG): container finished" podID="cfa0262c-6293-4621-81bb-fe494452904d" containerID="1d36eb6b2098473bdc2308c777a83537be342eec70466bbbe2c82f3b6d705c8d" exitCode=0
Jul 16 21:25:42.355445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:42.354763    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" event={"ID":"cfa0262c-6293-4621-81bb-fe494452904d","Type":"ContainerDied","Data":"1d36eb6b2098473bdc2308c777a83537be342eec70466bbbe2c82f3b6d705c8d"}
Jul 16 21:25:42.355445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:42.355212    2572 scope.go:117] "RemoveContainer" containerID="1d36eb6b2098473bdc2308c777a83537be342eec70466bbbe2c82f3b6d705c8d"
Jul 16 21:25:42.356282 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:42.356247    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:25:42.356369 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:42.356338    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:25:42.356423 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:42.356404    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:25:43.362601 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:43.362563    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" event={"ID":"cfa0262c-6293-4621-81bb-fe494452904d","Type":"ContainerStarted","Data":"1e9021a33dcbafb78389e5b5b6a312c18557384e7c08b91ecc3a4251ff90582d"}
Jul 16 21:25:44.327058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:44.327018    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:25:44.328332 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:44.328306    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xbb86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 16 21:25:44.328445 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:44.328404    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" podUID="cfa0262c-6293-4621-81bb-fe494452904d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 16 21:25:45.326848 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:45.326809    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xbb86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 16 21:25:45.327320 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:45.326867    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" podUID="cfa0262c-6293-4621-81bb-fe494452904d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 16 21:25:46.326707 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:46.326671    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xbb86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 16 21:25:46.326882 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:46.326726    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" podUID="cfa0262c-6293-4621-81bb-fe494452904d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 16 21:25:46.814534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:46.814493    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:25:47.326805 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:47.326769    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xbb86 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Jul 16 21:25:47.326987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:47.326833    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86" podUID="cfa0262c-6293-4621-81bb-fe494452904d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Jul 16 21:25:48.330956 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:48.330916    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:25:48.332067 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:48.332046    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xbb86"
Jul 16 21:25:59.832751 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.832670    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"]
Jul 16 21:25:59.835225 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.835207    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:25:59.838703 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.838682    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 21:25:59.838846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.838741    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5ktjw\""
Jul 16 21:25:59.838846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.838720    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 16 21:25:59.838846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.838714    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 21:25:59.845532 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.845508    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"]
Jul 16 21:25:59.915190 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.915158    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21e2154d-d9d3-4a27-aa99-ce472878334d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:25:59.915407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.915201    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6kkl\" (UniqueName: \"kubernetes.io/projected/21e2154d-d9d3-4a27-aa99-ce472878334d-kube-api-access-m6kkl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:25:59.915407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.915228    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:25:59.915407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.915253    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:25:59.915407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.915335    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:25:59.915551 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:25:59.915414    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.016568 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.016529    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21e2154d-d9d3-4a27-aa99-ce472878334d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.016740 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.016595    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6kkl\" (UniqueName: \"kubernetes.io/projected/21e2154d-d9d3-4a27-aa99-ce472878334d-kube-api-access-m6kkl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.016740 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.016623    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.016740 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.016647    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.016740 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.016722    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.016920 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.016767    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.017069 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.017048    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.017122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.017072    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.017160 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.017143    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.018868 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.018845    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21e2154d-d9d3-4a27-aa99-ce472878334d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.018974 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.018922    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21e2154d-d9d3-4a27-aa99-ce472878334d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.024868 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.024843    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6kkl\" (UniqueName: \"kubernetes.io/projected/21e2154d-d9d3-4a27-aa99-ce472878334d-kube-api-access-m6kkl\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n\" (UID: \"21e2154d-d9d3-4a27-aa99-ce472878334d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.148151 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.148065    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:00.292571 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.292539    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"]
Jul 16 21:26:00.293835 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:26:00.293803    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e2154d_d9d3_4a27_aa99_ce472878334d.slice/crio-af08701dae9c0589372c3d7198189e5ec55924115bca17ff938d1305fc9f5ae8 WatchSource:0}: Error finding container af08701dae9c0589372c3d7198189e5ec55924115bca17ff938d1305fc9f5ae8: Status 404 returned error can't find the container with id af08701dae9c0589372c3d7198189e5ec55924115bca17ff938d1305fc9f5ae8
Jul 16 21:26:00.295704 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.295669    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:26:00.457498 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:00.457407    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n" event={"ID":"21e2154d-d9d3-4a27-aa99-ce472878334d","Type":"ContainerStarted","Data":"af08701dae9c0589372c3d7198189e5ec55924115bca17ff938d1305fc9f5ae8"}
Jul 16 21:26:01.464165 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:01.464126    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n" event={"ID":"21e2154d-d9d3-4a27-aa99-ce472878334d","Type":"ContainerStarted","Data":"50327c8382e28dc40dcd4862b7a44d1b8f013c364c146c32b6fba0c3e12bdb3e"}
Jul 16 21:26:01.484764 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:01.484710    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n" podStartSLOduration=1.43654329 podStartE2EDuration="2.484691716s" totalImagesPullingTime="1.048148426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:26:01.482832084 +0000 UTC m=+626.993991816" watchObservedRunningTime="2026-07-16 21:26:01.484691716 +0000 UTC m=+626.995851456"
Jul 16 21:26:10.148274 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:10.148228    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:10.148877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:10.148305    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:10.161306 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:10.161277    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:10.523862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:10.523834    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n"
Jul 16 21:26:19.673723 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.673681    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"]
Jul 16 21:26:19.677623 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.677598    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.681055 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.681034    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 16 21:26:19.695330 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.695296    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"]
Jul 16 21:26:19.794117 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.794083    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.794117 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.794120    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.794338 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.794239    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.794338 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.794297    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n98p\" (UniqueName: \"kubernetes.io/projected/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-kube-api-access-8n98p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.794338 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.794333    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.794464 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.794363    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895341 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895294    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895341 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895345    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n98p\" (UniqueName: \"kubernetes.io/projected/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-kube-api-access-8n98p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895606 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895383    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895606 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895398    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895606 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895421    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895606 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895435    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895805 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895709    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895805 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895767    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.895873 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.895802    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.897738 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.897716    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.898010 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.897990    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.907391 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.907344    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n98p\" (UniqueName: \"kubernetes.io/projected/70730c39-d907-4f2c-8c20-81d2dcd6fc8d-kube-api-access-8n98p\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8vmdh\" (UID: \"70730c39-d907-4f2c-8c20-81d2dcd6fc8d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:19.988377 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:19.988310    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:20.131485 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:20.131456    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"]
Jul 16 21:26:20.133426 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:26:20.133393    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70730c39_d907_4f2c_8c20_81d2dcd6fc8d.slice/crio-9efc63a3e307c730f9efd500ff484fe682f6f4e766dda2ea9176fa4bdd7308c4 WatchSource:0}: Error finding container 9efc63a3e307c730f9efd500ff484fe682f6f4e766dda2ea9176fa4bdd7308c4: Status 404 returned error can't find the container with id 9efc63a3e307c730f9efd500ff484fe682f6f4e766dda2ea9176fa4bdd7308c4
Jul 16 21:26:20.565959 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:20.565920    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh" event={"ID":"70730c39-d907-4f2c-8c20-81d2dcd6fc8d","Type":"ContainerStarted","Data":"a4dd02184b643972c70d74219107655b70833bd1cdeba9b30bd4d470bf4d6cce"}
Jul 16 21:26:20.566154 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:20.565976    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh" event={"ID":"70730c39-d907-4f2c-8c20-81d2dcd6fc8d","Type":"ContainerStarted","Data":"9efc63a3e307c730f9efd500ff484fe682f6f4e766dda2ea9176fa4bdd7308c4"}
Jul 16 21:26:20.589546 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:20.589490    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh" podStartSLOduration=1.381338789 podStartE2EDuration="1.589471006s" totalImagesPullingTime="208.132217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:26:20.587276872 +0000 UTC m=+646.098436822" watchObservedRunningTime="2026-07-16 21:26:20.589471006 +0000 UTC m=+646.100630739"
Jul 16 21:26:29.989377 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:29.989308    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:29.989787 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:29.989398    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:30.002954 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:30.002922    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:26:30.630725 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:26:30.630692    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8vmdh"
Jul 16 21:29:16.295096 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.295059    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5659cfddb-6jgv6"]
Jul 16 21:29:16.295665 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.295287    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5659cfddb-6jgv6" podUID="5e214903-ae68-44f4-a559-3e93f69a2cc7" containerName="manager" containerID="cri-o://203aa6a548eda81f8c4dc68c8208d375ce1429d2ea54e0fc62963d10b5274f69" gracePeriod=30
Jul 16 21:29:16.534543 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.534516    2572 generic.go:356] "Generic (PLEG): container finished" podID="5e214903-ae68-44f4-a559-3e93f69a2cc7" containerID="203aa6a548eda81f8c4dc68c8208d375ce1429d2ea54e0fc62963d10b5274f69" exitCode=0
Jul 16 21:29:16.534658 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.534601    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-6jgv6" event={"ID":"5e214903-ae68-44f4-a559-3e93f69a2cc7","Type":"ContainerDied","Data":"203aa6a548eda81f8c4dc68c8208d375ce1429d2ea54e0fc62963d10b5274f69"}
Jul 16 21:29:16.549710 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.549645    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:29:16.655998 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.655960    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e214903-ae68-44f4-a559-3e93f69a2cc7-webhook-cert\") pod \"5e214903-ae68-44f4-a559-3e93f69a2cc7\" (UID: \"5e214903-ae68-44f4-a559-3e93f69a2cc7\") "
Jul 16 21:29:16.656175 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.656061    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwglq\" (UniqueName: \"kubernetes.io/projected/5e214903-ae68-44f4-a559-3e93f69a2cc7-kube-api-access-mwglq\") pod \"5e214903-ae68-44f4-a559-3e93f69a2cc7\" (UID: \"5e214903-ae68-44f4-a559-3e93f69a2cc7\") "
Jul 16 21:29:16.658293 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.658267    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e214903-ae68-44f4-a559-3e93f69a2cc7-kube-api-access-mwglq" (OuterVolumeSpecName: "kube-api-access-mwglq") pod "5e214903-ae68-44f4-a559-3e93f69a2cc7" (UID: "5e214903-ae68-44f4-a559-3e93f69a2cc7"). InnerVolumeSpecName "kube-api-access-mwglq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:29:16.658396 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.658267    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e214903-ae68-44f4-a559-3e93f69a2cc7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5e214903-ae68-44f4-a559-3e93f69a2cc7" (UID: "5e214903-ae68-44f4-a559-3e93f69a2cc7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:29:16.756858 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.756820    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwglq\" (UniqueName: \"kubernetes.io/projected/5e214903-ae68-44f4-a559-3e93f69a2cc7-kube-api-access-mwglq\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:29:16.756858 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:16.756852    2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5e214903-ae68-44f4-a559-3e93f69a2cc7-webhook-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:29:17.541147 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:17.541112    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-6jgv6" event={"ID":"5e214903-ae68-44f4-a559-3e93f69a2cc7","Type":"ContainerDied","Data":"668ce83fe46263e80c0493743f1971a4523508340b6e8d297853bcd5d1643f68"}
Jul 16 21:29:17.541147 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:17.541149    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-6jgv6"
Jul 16 21:29:17.541683 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:17.541183    2572 scope.go:117] "RemoveContainer" containerID="203aa6a548eda81f8c4dc68c8208d375ce1429d2ea54e0fc62963d10b5274f69"
Jul 16 21:29:17.561627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:17.561596    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5659cfddb-6jgv6"]
Jul 16 21:29:17.565835 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:17.565810    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5659cfddb-6jgv6"]
Jul 16 21:29:19.249984 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:29:19.249948    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e214903-ae68-44f4-a559-3e93f69a2cc7" path="/var/lib/kubelet/pods/5e214903-ae68-44f4-a559-3e93f69a2cc7/volumes"
Jul 16 21:30:00.205945 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.205905    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk"]
Jul 16 21:30:00.206571 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.206537    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e214903-ae68-44f4-a559-3e93f69a2cc7" containerName="manager"
Jul 16 21:30:00.206571 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.206558    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e214903-ae68-44f4-a559-3e93f69a2cc7" containerName="manager"
Jul 16 21:30:00.206701 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.206649    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e214903-ae68-44f4-a559-3e93f69a2cc7" containerName="manager"
Jul 16 21:30:00.210264 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.210235    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk"
Jul 16 21:30:00.213768 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.213746    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:30:00.223028 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.222995    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk"
Jul 16 21:30:00.224767 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.224739    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk"]
Jul 16 21:30:00.368851 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.368828    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk"]
Jul 16 21:30:00.370789 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:30:00.370762    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33a8a971_8011_4119_8e07_53c4a492ea44.slice/crio-a072d7b4790bd593b1787a8ca23a7bfbd1bc69bb04a84c0d20a867f0bcafa7b0 WatchSource:0}: Error finding container a072d7b4790bd593b1787a8ca23a7bfbd1bc69bb04a84c0d20a867f0bcafa7b0: Status 404 returned error can't find the container with id a072d7b4790bd593b1787a8ca23a7bfbd1bc69bb04a84c0d20a867f0bcafa7b0
Jul 16 21:30:00.781412 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:00.781373    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk" event={"ID":"33a8a971-8011-4119-8e07-53c4a492ea44","Type":"ContainerStarted","Data":"a072d7b4790bd593b1787a8ca23a7bfbd1bc69bb04a84c0d20a867f0bcafa7b0"}
Jul 16 21:30:03.799221 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:03.799184    2572 generic.go:356] "Generic (PLEG): container finished" podID="33a8a971-8011-4119-8e07-53c4a492ea44" containerID="9a061f42d3bf8a66299f1f24e12e5c386b6c29cef2b4cbeed69743a31f83a693" exitCode=0
Jul 16 21:30:03.799657 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:03.799273    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk" event={"ID":"33a8a971-8011-4119-8e07-53c4a492ea44","Type":"ContainerDied","Data":"9a061f42d3bf8a66299f1f24e12e5c386b6c29cef2b4cbeed69743a31f83a693"}
Jul 16 21:30:04.943070 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:04.943043    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk"
Jul 16 21:30:05.812702 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:05.812666    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk" event={"ID":"33a8a971-8011-4119-8e07-53c4a492ea44","Type":"ContainerDied","Data":"a072d7b4790bd593b1787a8ca23a7bfbd1bc69bb04a84c0d20a867f0bcafa7b0"}
Jul 16 21:30:05.812702 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:05.812688    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737290-s29qk"
Jul 16 21:30:05.812904 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:05.812720    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a072d7b4790bd593b1787a8ca23a7bfbd1bc69bb04a84c0d20a867f0bcafa7b0"
Jul 16 21:30:18.371193 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.371152    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5659cfddb-k5bzt"]
Jul 16 21:30:18.371758 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.371618    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33a8a971-8011-4119-8e07-53c4a492ea44" containerName="cleanup"
Jul 16 21:30:18.371758 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.371634    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="33a8a971-8011-4119-8e07-53c4a492ea44" containerName="cleanup"
Jul 16 21:30:18.371758 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.371700    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="33a8a971-8011-4119-8e07-53c4a492ea44" containerName="cleanup"
Jul 16 21:30:18.376901 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.376878    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:18.380373 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.380319    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 21:30:18.380373 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.380335    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 21:30:18.380570 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.380325    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mqzw5\""
Jul 16 21:30:18.389491 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.389464    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5659cfddb-k5bzt"]
Jul 16 21:30:18.436725 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.436560    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-webhook-cert\") pod \"maas-controller-5659cfddb-k5bzt\" (UID: \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\") " pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:18.436918 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.436731    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qsnl\" (UniqueName: \"kubernetes.io/projected/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-kube-api-access-7qsnl\") pod \"maas-controller-5659cfddb-k5bzt\" (UID: \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\") " pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:18.537757 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.537715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qsnl\" (UniqueName: \"kubernetes.io/projected/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-kube-api-access-7qsnl\") pod \"maas-controller-5659cfddb-k5bzt\" (UID: \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\") " pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:18.537939 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.537782    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-webhook-cert\") pod \"maas-controller-5659cfddb-k5bzt\" (UID: \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\") " pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:18.540268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.540239    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-webhook-cert\") pod \"maas-controller-5659cfddb-k5bzt\" (UID: \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\") " pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:18.545810 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.545787    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qsnl\" (UniqueName: \"kubernetes.io/projected/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-kube-api-access-7qsnl\") pod \"maas-controller-5659cfddb-k5bzt\" (UID: \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\") " pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:18.687582 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.687490    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:18.828166 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.828136    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5659cfddb-k5bzt"]
Jul 16 21:30:18.830077 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:30:18.830050    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b12d324_a0b0_4a4a_9afd_bcde5f4d3d21.slice/crio-cff31dbc88cd30ba0b773dbf4dda015b5f8562a5f92386fe5b4ad1b5a751de2f WatchSource:0}: Error finding container cff31dbc88cd30ba0b773dbf4dda015b5f8562a5f92386fe5b4ad1b5a751de2f: Status 404 returned error can't find the container with id cff31dbc88cd30ba0b773dbf4dda015b5f8562a5f92386fe5b4ad1b5a751de2f
Jul 16 21:30:18.883718 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:18.883682    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-k5bzt" event={"ID":"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21","Type":"ContainerStarted","Data":"cff31dbc88cd30ba0b773dbf4dda015b5f8562a5f92386fe5b4ad1b5a751de2f"}
Jul 16 21:30:19.891150 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:19.891115    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-k5bzt" event={"ID":"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21","Type":"ContainerStarted","Data":"0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959"}
Jul 16 21:30:19.891558 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:19.891177    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:19.912126 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:19.912045    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5659cfddb-k5bzt" podStartSLOduration=1.314345816 podStartE2EDuration="1.912027353s" totalImagesPullingTime="597.681537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:30:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:30:19.91031202 +0000 UTC m=+885.421471754" watchObservedRunningTime="2026-07-16 21:30:19.912027353 +0000 UTC m=+885.423187087"
Jul 16 21:30:30.902975 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:30.902886    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:35.530557 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.530521    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5659cfddb-k5bzt"]
Jul 16 21:30:35.531074 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.530760    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5659cfddb-k5bzt" podUID="8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21" containerName="manager" containerID="cri-o://0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959" gracePeriod=30
Jul 16 21:30:35.784374 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.784296    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:35.903912 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.903878    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-webhook-cert\") pod \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\" (UID: \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\") "
Jul 16 21:30:35.904090 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.903939    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qsnl\" (UniqueName: \"kubernetes.io/projected/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-kube-api-access-7qsnl\") pod \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\" (UID: \"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21\") "
Jul 16 21:30:35.905988 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.905955    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21" (UID: "8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:30:35.906102 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.906067    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-kube-api-access-7qsnl" (OuterVolumeSpecName: "kube-api-access-7qsnl") pod "8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21" (UID: "8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21"). InnerVolumeSpecName "kube-api-access-7qsnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:30:35.982010 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.981975    2572 generic.go:356] "Generic (PLEG): container finished" podID="8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21" containerID="0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959" exitCode=0
Jul 16 21:30:35.982174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.982026    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-k5bzt"
Jul 16 21:30:35.982174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.982061    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-k5bzt" event={"ID":"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21","Type":"ContainerDied","Data":"0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959"}
Jul 16 21:30:35.982174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.982116    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-k5bzt" event={"ID":"8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21","Type":"ContainerDied","Data":"cff31dbc88cd30ba0b773dbf4dda015b5f8562a5f92386fe5b4ad1b5a751de2f"}
Jul 16 21:30:35.982174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.982160    2572 scope.go:117] "RemoveContainer" containerID="0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959"
Jul 16 21:30:35.992215 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.992197    2572 scope.go:117] "RemoveContainer" containerID="0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959"
Jul 16 21:30:35.992545 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:30:35.992522    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959\": container with ID starting with 0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959 not found: ID does not exist" containerID="0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959"
Jul 16 21:30:35.992627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:35.992551    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959"} err="failed to get container status \"0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959\": rpc error: code = NotFound desc = could not find container \"0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959\": container with ID starting with 0b6ef92e03ef0216af8e40c0e835b0a870aa24a99e5c6e857064dd0b021eb959 not found: ID does not exist"
Jul 16 21:30:36.004844 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.004804    2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-webhook-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:30:36.004844 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.004841    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qsnl\" (UniqueName: \"kubernetes.io/projected/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21-kube-api-access-7qsnl\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:30:36.006888 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.006866    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5659cfddb-k5bzt"]
Jul 16 21:30:36.010634 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.010599    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5659cfddb-k5bzt"]
Jul 16 21:30:36.783721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.783689    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5659cfddb-8rcp4"]
Jul 16 21:30:36.784110 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.784086    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21" containerName="manager"
Jul 16 21:30:36.784110 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.784104    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21" containerName="manager"
Jul 16 21:30:36.784234 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.784187    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21" containerName="manager"
Jul 16 21:30:36.788532 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.788509    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:36.791141 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.791117    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 16 21:30:36.792069 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.792050    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-mqzw5\""
Jul 16 21:30:36.792214 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.792073    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 16 21:30:36.799149 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.799120    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5659cfddb-8rcp4"]
Jul 16 21:30:36.914906 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.914868    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nncxb\" (UniqueName: \"kubernetes.io/projected/a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a-kube-api-access-nncxb\") pod \"maas-controller-5659cfddb-8rcp4\" (UID: \"a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a\") " pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:36.915101 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:36.914947    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a-webhook-cert\") pod \"maas-controller-5659cfddb-8rcp4\" (UID: \"a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a\") " pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:37.016004 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.015967    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a-webhook-cert\") pod \"maas-controller-5659cfddb-8rcp4\" (UID: \"a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a\") " pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:37.016189 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.016051    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nncxb\" (UniqueName: \"kubernetes.io/projected/a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a-kube-api-access-nncxb\") pod \"maas-controller-5659cfddb-8rcp4\" (UID: \"a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a\") " pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:37.018487 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.018468    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a-webhook-cert\") pod \"maas-controller-5659cfddb-8rcp4\" (UID: \"a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a\") " pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:37.026311 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.026287    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nncxb\" (UniqueName: \"kubernetes.io/projected/a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a-kube-api-access-nncxb\") pod \"maas-controller-5659cfddb-8rcp4\" (UID: \"a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a\") " pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:37.101429 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.101315    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:37.242892 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.242862    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5659cfddb-8rcp4"]
Jul 16 21:30:37.245345 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:30:37.245313    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49f83a4_d0d1_4892_a665_1fd5bdbfaf4a.slice/crio-689c7d5ad415f53d0a29074d8b28ef17dc6c3c01424a8a201d27bf832ba00939 WatchSource:0}: Error finding container 689c7d5ad415f53d0a29074d8b28ef17dc6c3c01424a8a201d27bf832ba00939: Status 404 returned error can't find the container with id 689c7d5ad415f53d0a29074d8b28ef17dc6c3c01424a8a201d27bf832ba00939
Jul 16 21:30:37.251458 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.251412    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21" path="/var/lib/kubelet/pods/8b12d324-a0b0-4a4a-9afd-bcde5f4d3d21/volumes"
Jul 16 21:30:37.996107 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.996070    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-8rcp4" event={"ID":"a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a","Type":"ContainerStarted","Data":"d278d452ce175f93bcd5c557588da0f466c0a23440ca5c7f15d9e07884799b4d"}
Jul 16 21:30:37.996536 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.996120    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5659cfddb-8rcp4" event={"ID":"a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a","Type":"ContainerStarted","Data":"689c7d5ad415f53d0a29074d8b28ef17dc6c3c01424a8a201d27bf832ba00939"}
Jul 16 21:30:37.996536 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:37.996194    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:30:38.017342 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:38.017283    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5659cfddb-8rcp4" podStartSLOduration=1.437716578 podStartE2EDuration="2.017264899s" totalImagesPullingTime="579.548321ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:30:38.01516079 +0000 UTC m=+903.526320518" watchObservedRunningTime="2026-07-16 21:30:38.017264899 +0000 UTC m=+903.528424633"
Jul 16 21:30:49.003888 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:30:49.003857    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5659cfddb-8rcp4"
Jul 16 21:42:48.270998 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.270814    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"]
Jul 16 21:42:48.274811 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.274788    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.277333 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.277314    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fa7d409d-openshift-default-dockercfg-k6hmf\""
Jul 16 21:42:48.298286 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.298244    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"]
Jul 16 21:42:48.370583 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.370546    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-envoy\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.370780 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.370591    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-socket\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.370780 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.370691    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-podinfo\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.370780 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.370724    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-credential-socket\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.370780 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.370747    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-data\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.370961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.370832    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpdmd\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-kube-api-access-dpdmd\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.370961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.370887    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-certs\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.371029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.370966    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-token\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.371029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.371021    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istiod-ca-cert\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.471529 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471490    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-data\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.471529 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471536    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpdmd\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-kube-api-access-dpdmd\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.471783 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471562    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-certs\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.471783 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471596    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-token\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.471783 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471628    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istiod-ca-cert\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.471783 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471662    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-envoy\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.472076 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471818    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-socket\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.472076 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471891    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-podinfo\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.472076 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471928    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-credential-socket\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.472076 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.471989    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-data\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.472076 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.472036    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-certs\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.472390 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.472212    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-socket\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.472390 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.472274    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-credential-socket\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.472523 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.472485    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istiod-ca-cert\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.473983 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.473966    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-envoy\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.474294 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.474278    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-podinfo\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.480033 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.480004    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-token\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.480186 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.480045    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpdmd\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-kube-api-access-dpdmd\") pod \"e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") " pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.585575 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.585529    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:48.730547 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.730520    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"]
Jul 16 21:42:48.732976 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:42:48.732949    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode903ea0c_511a_49be_aa9d_aabe4e0601b6.slice/crio-5a0fb6edba60cbd5dffd874594c96e3be2d3472d92b6b1a45e814991f78632d9 WatchSource:0}: Error finding container 5a0fb6edba60cbd5dffd874594c96e3be2d3472d92b6b1a45e814991f78632d9: Status 404 returned error can't find the container with id 5a0fb6edba60cbd5dffd874594c96e3be2d3472d92b6b1a45e814991f78632d9
Jul 16 21:42:48.735267 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.735224    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:42:48.735366 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.735318    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:42:48.735419 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.735377    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:42:48.990504 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.990412    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" event={"ID":"e903ea0c-511a-49be-aa9d-aabe4e0601b6","Type":"ContainerStarted","Data":"f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2"}
Jul 16 21:42:48.990504 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:48.990465    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" event={"ID":"e903ea0c-511a-49be-aa9d-aabe4e0601b6","Type":"ContainerStarted","Data":"5a0fb6edba60cbd5dffd874594c96e3be2d3472d92b6b1a45e814991f78632d9"}
Jul 16 21:42:49.012154 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:49.012104    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" podStartSLOduration=1.012086255 podStartE2EDuration="1.012086255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:42:49.008898768 +0000 UTC m=+1634.520058507" watchObservedRunningTime="2026-07-16 21:42:49.012086255 +0000 UTC m=+1634.523245991"
Jul 16 21:42:49.586563 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:49.586517    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:49.593862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:49.593835    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:49.996308 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:49.996221    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:49.997217 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:49.997194    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:42:54.430223 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.430185    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"]
Jul 16 21:42:54.433980 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.433963    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"
Jul 16 21:42:54.436278 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.436260    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:42:54.441684 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.441653    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"]
Jul 16 21:42:54.531590 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.531545    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54d101b-12c5-4cd3-9cee-341c583211b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv\" (UID: \"a54d101b-12c5-4cd3-9cee-341c583211b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"
Jul 16 21:42:54.632881 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.632837    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54d101b-12c5-4cd3-9cee-341c583211b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv\" (UID: \"a54d101b-12c5-4cd3-9cee-341c583211b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"
Jul 16 21:42:54.633517 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.633496    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54d101b-12c5-4cd3-9cee-341c583211b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv\" (UID: \"a54d101b-12c5-4cd3-9cee-341c583211b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"
Jul 16 21:42:54.747961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.747877    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"
Jul 16 21:42:54.898545 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:54.898516    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"]
Jul 16 21:42:54.900946 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:42:54.900917    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda54d101b_12c5_4cd3_9cee_341c583211b6.slice/crio-622566ac79fabeb17f86e230984983ce98652f5a29e2e221a28a147dd861c8fd WatchSource:0}: Error finding container 622566ac79fabeb17f86e230984983ce98652f5a29e2e221a28a147dd861c8fd: Status 404 returned error can't find the container with id 622566ac79fabeb17f86e230984983ce98652f5a29e2e221a28a147dd861c8fd
Jul 16 21:42:55.024429 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:55.024396    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv" event={"ID":"a54d101b-12c5-4cd3-9cee-341c583211b6","Type":"ContainerStarted","Data":"622566ac79fabeb17f86e230984983ce98652f5a29e2e221a28a147dd861c8fd"}
Jul 16 21:42:56.031161 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:56.031130    2572 generic.go:356] "Generic (PLEG): container finished" podID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerID="6cd0f0375fc7e2ba0aafb14787c032a291e2af468ccd36174a42f34bcd582a23" exitCode=7
Jul 16 21:42:56.031638 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:56.031212    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv" event={"ID":"a54d101b-12c5-4cd3-9cee-341c583211b6","Type":"ContainerDied","Data":"6cd0f0375fc7e2ba0aafb14787c032a291e2af468ccd36174a42f34bcd582a23"}
Jul 16 21:42:56.031638 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:56.031416    2572 scope.go:117] "RemoveContainer" containerID="6cd0f0375fc7e2ba0aafb14787c032a291e2af468ccd36174a42f34bcd582a23"
Jul 16 21:42:57.039718 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:57.039679    2572 generic.go:356] "Generic (PLEG): container finished" podID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerID="ae439b2cc8ff379a6fdce2ff23e5555c011b804aae9fd937c47b8eadb95cd5db" exitCode=7
Jul 16 21:42:57.040197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:57.039769    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv" event={"ID":"a54d101b-12c5-4cd3-9cee-341c583211b6","Type":"ContainerDied","Data":"ae439b2cc8ff379a6fdce2ff23e5555c011b804aae9fd937c47b8eadb95cd5db"}
Jul 16 21:42:57.040197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:57.039830    2572 scope.go:117] "RemoveContainer" containerID="6cd0f0375fc7e2ba0aafb14787c032a291e2af468ccd36174a42f34bcd582a23"
Jul 16 21:42:57.040197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:57.040039    2572 scope.go:117] "RemoveContainer" containerID="ae439b2cc8ff379a6fdce2ff23e5555c011b804aae9fd937c47b8eadb95cd5db"
Jul 16 21:42:57.040388 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:42:57.040301    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv_odh-ai-gateway-infra(a54d101b-12c5-4cd3-9cee-341c583211b6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6"
Jul 16 21:42:58.047220 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:42:58.047191    2572 scope.go:117] "RemoveContainer" containerID="ae439b2cc8ff379a6fdce2ff23e5555c011b804aae9fd937c47b8eadb95cd5db"
Jul 16 21:42:58.047640 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:42:58.047422    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv_odh-ai-gateway-infra(a54d101b-12c5-4cd3-9cee-341c583211b6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6"
Jul 16 21:43:04.997429 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:04.997315    2572 patch_prober.go:28] interesting pod/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 21:43:04.997429 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:04.997387    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 21:43:05.088786 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:05.088753    2572 generic.go:356] "Generic (PLEG): container finished" podID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerID="f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2" exitCode=0
Jul 16 21:43:05.088956 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:05.088829    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" event={"ID":"e903ea0c-511a-49be-aa9d-aabe4e0601b6","Type":"ContainerDied","Data":"f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2"}
Jul 16 21:43:05.089315 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:05.089298    2572 scope.go:117] "RemoveContainer" containerID="f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2"
Jul 16 21:43:05.090375 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:05.090331    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:43:05.090481 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:05.090430    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:43:05.090543 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:05.090481    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:43:06.100431 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:06.100385    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" event={"ID":"e903ea0c-511a-49be-aa9d-aabe4e0601b6","Type":"ContainerStarted","Data":"adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423"}
Jul 16 21:43:06.587065 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:06.587015    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:43:06.587257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:06.587168    2572 patch_prober.go:28] interesting pod/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 21:43:06.587257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:06.587217    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 21:43:07.585982 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:07.585932    2572 patch_prober.go:28] interesting pod/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 21:43:07.586538 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:07.586017    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 21:43:08.586395 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:08.586324    2572 patch_prober.go:28] interesting pod/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 21:43:08.586927 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:08.586435    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 21:43:09.586050 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:09.586001    2572 patch_prober.go:28] interesting pod/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 16 21:43:09.586266 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:09.586071    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 16 21:43:10.590251 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:10.590220    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:43:10.590669 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:10.590567    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:43:10.591190 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:10.591169    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:43:12.243857 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:12.243818    2572 scope.go:117] "RemoveContainer" containerID="ae439b2cc8ff379a6fdce2ff23e5555c011b804aae9fd937c47b8eadb95cd5db"
Jul 16 21:43:13.147941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:13.147850    2572 generic.go:356] "Generic (PLEG): container finished" podID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerID="c7e393b08a228054c44f976798f6553379acb83868f60c4a39e12dad7a314e93" exitCode=7
Jul 16 21:43:13.147941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:13.147907    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv" event={"ID":"a54d101b-12c5-4cd3-9cee-341c583211b6","Type":"ContainerDied","Data":"c7e393b08a228054c44f976798f6553379acb83868f60c4a39e12dad7a314e93"}
Jul 16 21:43:13.148197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:13.147955    2572 scope.go:117] "RemoveContainer" containerID="ae439b2cc8ff379a6fdce2ff23e5555c011b804aae9fd937c47b8eadb95cd5db"
Jul 16 21:43:13.148344 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:13.148326    2572 scope.go:117] "RemoveContainer" containerID="c7e393b08a228054c44f976798f6553379acb83868f60c4a39e12dad7a314e93"
Jul 16 21:43:13.148622 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:43:13.148600    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv_odh-ai-gateway-infra(a54d101b-12c5-4cd3-9cee-341c583211b6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6"
Jul 16 21:43:14.183029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:14.182983    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"]
Jul 16 21:43:14.322143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:14.322119    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"
Jul 16 21:43:14.338460 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:14.338429    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54d101b-12c5-4cd3-9cee-341c583211b6-maas-api-service-ca\") pod \"a54d101b-12c5-4cd3-9cee-341c583211b6\" (UID: \"a54d101b-12c5-4cd3-9cee-341c583211b6\") "
Jul 16 21:43:14.338886 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:14.338855    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a54d101b-12c5-4cd3-9cee-341c583211b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a54d101b-12c5-4cd3-9cee-341c583211b6" (UID: "a54d101b-12c5-4cd3-9cee-341c583211b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:43:14.439868 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:14.439782    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a54d101b-12c5-4cd3-9cee-341c583211b6-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:43:15.161226 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:15.161198    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"
Jul 16 21:43:15.161422 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:15.161201    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv" event={"ID":"a54d101b-12c5-4cd3-9cee-341c583211b6","Type":"ContainerDied","Data":"622566ac79fabeb17f86e230984983ce98652f5a29e2e221a28a147dd861c8fd"}
Jul 16 21:43:15.161422 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:15.161334    2572 scope.go:117] "RemoveContainer" containerID="c7e393b08a228054c44f976798f6553379acb83868f60c4a39e12dad7a314e93"
Jul 16 21:43:15.201794 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:15.201754    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"]
Jul 16 21:43:15.224300 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:15.224265    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-lqssv"]
Jul 16 21:43:15.249423 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:15.249391    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" path="/var/lib/kubelet/pods/a54d101b-12c5-4cd3-9cee-341c583211b6/volumes"
Jul 16 21:43:54.508438 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508402    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"]
Jul 16 21:43:54.508877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508791    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:43:54.508877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508802    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:43:54.508877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508824    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:43:54.508877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508830    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:43:54.508877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508838    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:43:54.508877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508843    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:43:54.509092 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508904    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:43:54.509092 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.508913    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:43:54.511590 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.511571    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"
Jul 16 21:43:54.514331 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.514307    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:43:54.521763 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.521740    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2060b18-2b32-4e69-84cf-69a226cdfc66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9\" (UID: \"d2060b18-2b32-4e69-84cf-69a226cdfc66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"
Jul 16 21:43:54.529530 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.529502    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"]
Jul 16 21:43:54.623109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.623059    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2060b18-2b32-4e69-84cf-69a226cdfc66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9\" (UID: \"d2060b18-2b32-4e69-84cf-69a226cdfc66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"
Jul 16 21:43:54.623812 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.623791    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2060b18-2b32-4e69-84cf-69a226cdfc66-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9\" (UID: \"d2060b18-2b32-4e69-84cf-69a226cdfc66\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"
Jul 16 21:43:54.821694 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.821661    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"
Jul 16 21:43:54.969044 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:54.969016    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"]
Jul 16 21:43:54.970632 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:43:54.970598    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2060b18_2b32_4e69_84cf_69a226cdfc66.slice/crio-bf2f6522e82a71145ffdeed325d61d18656522e76710636e8c7d073afe1cd387 WatchSource:0}: Error finding container bf2f6522e82a71145ffdeed325d61d18656522e76710636e8c7d073afe1cd387: Status 404 returned error can't find the container with id bf2f6522e82a71145ffdeed325d61d18656522e76710636e8c7d073afe1cd387
Jul 16 21:43:55.388520 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:55.388483    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" event={"ID":"d2060b18-2b32-4e69-84cf-69a226cdfc66","Type":"ContainerStarted","Data":"e2ac7458bc796af2b5a8ffa7001944a08288a59dfda62095028ad6b73bc63e41"}
Jul 16 21:43:55.388694 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:55.388537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" event={"ID":"d2060b18-2b32-4e69-84cf-69a226cdfc66","Type":"ContainerStarted","Data":"bf2f6522e82a71145ffdeed325d61d18656522e76710636e8c7d073afe1cd387"}
Jul 16 21:43:55.444655 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:55.444596    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" podStartSLOduration=1.44457664 podStartE2EDuration="1.44457664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:43:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:43:55.4420908 +0000 UTC m=+1700.953250533" watchObservedRunningTime="2026-07-16 21:43:55.44457664 +0000 UTC m=+1700.955736374"
Jul 16 21:43:56.396064 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:56.395984    2572 generic.go:356] "Generic (PLEG): container finished" podID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerID="e2ac7458bc796af2b5a8ffa7001944a08288a59dfda62095028ad6b73bc63e41" exitCode=7
Jul 16 21:43:56.396470 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:56.396074    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" event={"ID":"d2060b18-2b32-4e69-84cf-69a226cdfc66","Type":"ContainerDied","Data":"e2ac7458bc796af2b5a8ffa7001944a08288a59dfda62095028ad6b73bc63e41"}
Jul 16 21:43:56.396470 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:56.396260    2572 scope.go:117] "RemoveContainer" containerID="e2ac7458bc796af2b5a8ffa7001944a08288a59dfda62095028ad6b73bc63e41"
Jul 16 21:43:57.403865 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:57.403835    2572 generic.go:356] "Generic (PLEG): container finished" podID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerID="d52bc841d567bd4d3a6964489c80dcd92b66d96afe040e65f806ddab80d35813" exitCode=7
Jul 16 21:43:57.404285 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:57.403917    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" event={"ID":"d2060b18-2b32-4e69-84cf-69a226cdfc66","Type":"ContainerDied","Data":"d52bc841d567bd4d3a6964489c80dcd92b66d96afe040e65f806ddab80d35813"}
Jul 16 21:43:57.404285 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:57.403975    2572 scope.go:117] "RemoveContainer" containerID="e2ac7458bc796af2b5a8ffa7001944a08288a59dfda62095028ad6b73bc63e41"
Jul 16 21:43:57.404285 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:57.404209    2572 scope.go:117] "RemoveContainer" containerID="d52bc841d567bd4d3a6964489c80dcd92b66d96afe040e65f806ddab80d35813"
Jul 16 21:43:57.404496 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:43:57.404477    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9_odh-ai-gateway-infra(d2060b18-2b32-4e69-84cf-69a226cdfc66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66"
Jul 16 21:43:58.410900 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:43:58.410873    2572 scope.go:117] "RemoveContainer" containerID="d52bc841d567bd4d3a6964489c80dcd92b66d96afe040e65f806ddab80d35813"
Jul 16 21:43:58.411283 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:43:58.411081    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9_odh-ai-gateway-infra(d2060b18-2b32-4e69-84cf-69a226cdfc66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66"
Jul 16 21:44:10.244090 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:10.244054    2572 scope.go:117] "RemoveContainer" containerID="d52bc841d567bd4d3a6964489c80dcd92b66d96afe040e65f806ddab80d35813"
Jul 16 21:44:11.482800 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:11.482764    2572 generic.go:356] "Generic (PLEG): container finished" podID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerID="0c58be5902e73c4fe7ab6b1c6f3bb507ebcd6a5f4364c78e61c32421e1b905d9" exitCode=7
Jul 16 21:44:11.483268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:11.482830    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" event={"ID":"d2060b18-2b32-4e69-84cf-69a226cdfc66","Type":"ContainerDied","Data":"0c58be5902e73c4fe7ab6b1c6f3bb507ebcd6a5f4364c78e61c32421e1b905d9"}
Jul 16 21:44:11.483268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:11.482889    2572 scope.go:117] "RemoveContainer" containerID="d52bc841d567bd4d3a6964489c80dcd92b66d96afe040e65f806ddab80d35813"
Jul 16 21:44:11.483268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:11.483192    2572 scope.go:117] "RemoveContainer" containerID="0c58be5902e73c4fe7ab6b1c6f3bb507ebcd6a5f4364c78e61c32421e1b905d9"
Jul 16 21:44:11.483464 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:44:11.483425    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9_odh-ai-gateway-infra(d2060b18-2b32-4e69-84cf-69a226cdfc66)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66"
Jul 16 21:44:12.539341 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:12.539303    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"]
Jul 16 21:44:12.678910 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:12.678882    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"
Jul 16 21:44:12.797770 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:12.797681    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2060b18-2b32-4e69-84cf-69a226cdfc66-maas-api-service-ca\") pod \"d2060b18-2b32-4e69-84cf-69a226cdfc66\" (UID: \"d2060b18-2b32-4e69-84cf-69a226cdfc66\") "
Jul 16 21:44:12.798053 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:12.798030    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2060b18-2b32-4e69-84cf-69a226cdfc66-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d2060b18-2b32-4e69-84cf-69a226cdfc66" (UID: "d2060b18-2b32-4e69-84cf-69a226cdfc66"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:44:12.898509 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:12.898469    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2060b18-2b32-4e69-84cf-69a226cdfc66-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:44:13.497417 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:13.497383    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9" event={"ID":"d2060b18-2b32-4e69-84cf-69a226cdfc66","Type":"ContainerDied","Data":"bf2f6522e82a71145ffdeed325d61d18656522e76710636e8c7d073afe1cd387"}
Jul 16 21:44:13.497417 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:13.497406    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"
Jul 16 21:44:13.497653 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:13.497452    2572 scope.go:117] "RemoveContainer" containerID="0c58be5902e73c4fe7ab6b1c6f3bb507ebcd6a5f4364c78e61c32421e1b905d9"
Jul 16 21:44:13.544283 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:13.544235    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"]
Jul 16 21:44:13.544906 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:13.544885    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-pl9l9"]
Jul 16 21:44:15.250998 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:15.250967    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" path="/var/lib/kubelet/pods/d2060b18-2b32-4e69-84cf-69a226cdfc66/volumes"
Jul 16 21:44:44.586135 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586090    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"]
Jul 16 21:44:44.586787 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586718    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:44:44.586787 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586741    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:44:44.586909 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586805    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:44:44.586909 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586814    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:44:44.587007 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586909    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a54d101b-12c5-4cd3-9cee-341c583211b6" containerName="revoke-keys"
Jul 16 21:44:44.587007 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586926    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:44:44.587007 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586939    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:44:44.587007 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.586948    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:44:44.589573 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.589550    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"
Jul 16 21:44:44.593394 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.593373    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:44:44.609148 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.609117    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"]
Jul 16 21:44:44.701831 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.701787    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7c8c38-4b98-41b7-81dc-8add79435b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8\" (UID: \"ae7c8c38-4b98-41b7-81dc-8add79435b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"
Jul 16 21:44:44.803405 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.803347    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7c8c38-4b98-41b7-81dc-8add79435b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8\" (UID: \"ae7c8c38-4b98-41b7-81dc-8add79435b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"
Jul 16 21:44:44.804096 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.804073    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7c8c38-4b98-41b7-81dc-8add79435b63-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8\" (UID: \"ae7c8c38-4b98-41b7-81dc-8add79435b63\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"
Jul 16 21:44:44.901243 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:44.901146    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"
Jul 16 21:44:45.079931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:45.079890    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"]
Jul 16 21:44:45.084270 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:44:45.084238    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae7c8c38_4b98_41b7_81dc_8add79435b63.slice/crio-521c11e179d38064a737a49c470ab158b1b4e3e39ba585f21d812180487dfb12 WatchSource:0}: Error finding container 521c11e179d38064a737a49c470ab158b1b4e3e39ba585f21d812180487dfb12: Status 404 returned error can't find the container with id 521c11e179d38064a737a49c470ab158b1b4e3e39ba585f21d812180487dfb12
Jul 16 21:44:45.682864 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:45.682762    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" event={"ID":"ae7c8c38-4b98-41b7-81dc-8add79435b63","Type":"ContainerStarted","Data":"0ea092ef90b16aceca77596d642bb59445296088fc6e3cf54541da07343650a1"}
Jul 16 21:44:45.682864 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:45.682820    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" event={"ID":"ae7c8c38-4b98-41b7-81dc-8add79435b63","Type":"ContainerStarted","Data":"521c11e179d38064a737a49c470ab158b1b4e3e39ba585f21d812180487dfb12"}
Jul 16 21:44:45.728038 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:45.727978    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" podStartSLOduration=1.727956434 podStartE2EDuration="1.727956434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:44:45.727342404 +0000 UTC m=+1751.238502138" watchObservedRunningTime="2026-07-16 21:44:45.727956434 +0000 UTC m=+1751.239116170"
Jul 16 21:44:46.690068 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:46.690037    2572 generic.go:356] "Generic (PLEG): container finished" podID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerID="0ea092ef90b16aceca77596d642bb59445296088fc6e3cf54541da07343650a1" exitCode=7
Jul 16 21:44:46.690526 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:46.690123    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" event={"ID":"ae7c8c38-4b98-41b7-81dc-8add79435b63","Type":"ContainerDied","Data":"0ea092ef90b16aceca77596d642bb59445296088fc6e3cf54541da07343650a1"}
Jul 16 21:44:46.690526 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:46.690318    2572 scope.go:117] "RemoveContainer" containerID="0ea092ef90b16aceca77596d642bb59445296088fc6e3cf54541da07343650a1"
Jul 16 21:44:47.697832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:47.697741    2572 generic.go:356] "Generic (PLEG): container finished" podID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerID="236b5fdda4f0b62bf1fbd3bf3ccb259e8a1fba99476f1c3e39ad292ebdfa6f3c" exitCode=7
Jul 16 21:44:47.697832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:47.697792    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" event={"ID":"ae7c8c38-4b98-41b7-81dc-8add79435b63","Type":"ContainerDied","Data":"236b5fdda4f0b62bf1fbd3bf3ccb259e8a1fba99476f1c3e39ad292ebdfa6f3c"}
Jul 16 21:44:47.698318 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:47.697841    2572 scope.go:117] "RemoveContainer" containerID="0ea092ef90b16aceca77596d642bb59445296088fc6e3cf54541da07343650a1"
Jul 16 21:44:47.698318 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:47.698096    2572 scope.go:117] "RemoveContainer" containerID="236b5fdda4f0b62bf1fbd3bf3ccb259e8a1fba99476f1c3e39ad292ebdfa6f3c"
Jul 16 21:44:47.698439 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:44:47.698316    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8_odh-ai-gateway-infra(ae7c8c38-4b98-41b7-81dc-8add79435b63)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63"
Jul 16 21:44:48.705085 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:44:48.705062    2572 scope.go:117] "RemoveContainer" containerID="236b5fdda4f0b62bf1fbd3bf3ccb259e8a1fba99476f1c3e39ad292ebdfa6f3c"
Jul 16 21:44:48.705588 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:44:48.705284    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8_odh-ai-gateway-infra(ae7c8c38-4b98-41b7-81dc-8add79435b63)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63"
Jul 16 21:45:00.173492 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.173462    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"]
Jul 16 21:45:00.173870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.173860    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:45:00.173913 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.173871    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2060b18-2b32-4e69-84cf-69a226cdfc66" containerName="revoke-keys"
Jul 16 21:45:00.176193 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.176178    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"
Jul 16 21:45:00.185483 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.185464    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"
Jul 16 21:45:00.210182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.210135    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"]
Jul 16 21:45:00.330757 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.330727    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"]
Jul 16 21:45:00.332194 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:45:00.332169    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9815a822_5f4d_4302_a6cf_b54f8a0a22cb.slice/crio-ad762d1ec6726bc76deb67b00fec00c39a074636c59cbd93f1cb2c32576b9041 WatchSource:0}: Error finding container ad762d1ec6726bc76deb67b00fec00c39a074636c59cbd93f1cb2c32576b9041: Status 404 returned error can't find the container with id ad762d1ec6726bc76deb67b00fec00c39a074636c59cbd93f1cb2c32576b9041
Jul 16 21:45:00.772666 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.772632    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" event={"ID":"9815a822-5f4d-4302-a6cf-b54f8a0a22cb","Type":"ContainerStarted","Data":"fd196e96229a93ab3352caf365e98bcdf27e83a6f23f68535ac471deddcf5a27"}
Jul 16 21:45:00.772826 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:00.772692    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" event={"ID":"9815a822-5f4d-4302-a6cf-b54f8a0a22cb","Type":"ContainerStarted","Data":"ad762d1ec6726bc76deb67b00fec00c39a074636c59cbd93f1cb2c32576b9041"}
Jul 16 21:45:01.243901 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:01.243820    2572 scope.go:117] "RemoveContainer" containerID="236b5fdda4f0b62bf1fbd3bf3ccb259e8a1fba99476f1c3e39ad292ebdfa6f3c"
Jul 16 21:45:01.779774 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:01.779745    2572 generic.go:356] "Generic (PLEG): container finished" podID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerID="fd196e96229a93ab3352caf365e98bcdf27e83a6f23f68535ac471deddcf5a27" exitCode=7
Jul 16 21:45:01.779985 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:01.779841    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" event={"ID":"9815a822-5f4d-4302-a6cf-b54f8a0a22cb","Type":"ContainerDied","Data":"fd196e96229a93ab3352caf365e98bcdf27e83a6f23f68535ac471deddcf5a27"}
Jul 16 21:45:01.780056 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:01.779982    2572 scope.go:117] "RemoveContainer" containerID="fd196e96229a93ab3352caf365e98bcdf27e83a6f23f68535ac471deddcf5a27"
Jul 16 21:45:01.781856 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:01.781833    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" event={"ID":"ae7c8c38-4b98-41b7-81dc-8add79435b63","Type":"ContainerStarted","Data":"fce11b4713dedca33b031731b26a05b980864e18ea08085156816f0d7ab8b308"}
Jul 16 21:45:02.789609 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.789576    2572 generic.go:356] "Generic (PLEG): container finished" podID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerID="5c8ab253e489c6670322489bf205725a760cdeef66c02e5f2e77c223e4a38435" exitCode=7
Jul 16 21:45:02.790058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.789668    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" event={"ID":"9815a822-5f4d-4302-a6cf-b54f8a0a22cb","Type":"ContainerDied","Data":"5c8ab253e489c6670322489bf205725a760cdeef66c02e5f2e77c223e4a38435"}
Jul 16 21:45:02.790058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.789739    2572 scope.go:117] "RemoveContainer" containerID="fd196e96229a93ab3352caf365e98bcdf27e83a6f23f68535ac471deddcf5a27"
Jul 16 21:45:02.790058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.789819    2572 scope.go:117] "RemoveContainer" containerID="5c8ab253e489c6670322489bf205725a760cdeef66c02e5f2e77c223e4a38435"
Jul 16 21:45:02.790202 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:45:02.790066    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9_odh-ai-gateway-infra(9815a822-5f4d-4302-a6cf-b54f8a0a22cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb"
Jul 16 21:45:02.791642 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.791625    2572 generic.go:356] "Generic (PLEG): container finished" podID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerID="fce11b4713dedca33b031731b26a05b980864e18ea08085156816f0d7ab8b308" exitCode=7
Jul 16 21:45:02.791737 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.791693    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" event={"ID":"ae7c8c38-4b98-41b7-81dc-8add79435b63","Type":"ContainerDied","Data":"fce11b4713dedca33b031731b26a05b980864e18ea08085156816f0d7ab8b308"}
Jul 16 21:45:02.791943 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.791930    2572 scope.go:117] "RemoveContainer" containerID="fce11b4713dedca33b031731b26a05b980864e18ea08085156816f0d7ab8b308"
Jul 16 21:45:02.792092 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:45:02.792077    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8_odh-ai-gateway-infra(ae7c8c38-4b98-41b7-81dc-8add79435b63)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63"
Jul 16 21:45:02.800411 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.800392    2572 scope.go:117] "RemoveContainer" containerID="236b5fdda4f0b62bf1fbd3bf3ccb259e8a1fba99476f1c3e39ad292ebdfa6f3c"
Jul 16 21:45:02.865067 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:02.859787    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"]
Jul 16 21:45:03.801544 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:03.801522    2572 scope.go:117] "RemoveContainer" containerID="5c8ab253e489c6670322489bf205725a760cdeef66c02e5f2e77c223e4a38435"
Jul 16 21:45:03.801865 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:45:03.801739    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9_odh-ai-gateway-infra(9815a822-5f4d-4302-a6cf-b54f8a0a22cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb"
Jul 16 21:45:03.927406 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:03.927376    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"
Jul 16 21:45:03.999783 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:03.999756    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7c8c38-4b98-41b7-81dc-8add79435b63-maas-api-service-ca\") pod \"ae7c8c38-4b98-41b7-81dc-8add79435b63\" (UID: \"ae7c8c38-4b98-41b7-81dc-8add79435b63\") "
Jul 16 21:45:04.000091 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:04.000069    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae7c8c38-4b98-41b7-81dc-8add79435b63-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae7c8c38-4b98-41b7-81dc-8add79435b63" (UID: "ae7c8c38-4b98-41b7-81dc-8add79435b63"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:45:04.100994 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:04.100896    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae7c8c38-4b98-41b7-81dc-8add79435b63-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:45:04.808780 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:04.808750    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"
Jul 16 21:45:04.809182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:04.808784    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8" event={"ID":"ae7c8c38-4b98-41b7-81dc-8add79435b63","Type":"ContainerDied","Data":"521c11e179d38064a737a49c470ab158b1b4e3e39ba585f21d812180487dfb12"}
Jul 16 21:45:04.809182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:04.808852    2572 scope.go:117] "RemoveContainer" containerID="fce11b4713dedca33b031731b26a05b980864e18ea08085156816f0d7ab8b308"
Jul 16 21:45:04.849236 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:04.849198    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"]
Jul 16 21:45:04.859174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:04.859146    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-z9ff8"]
Jul 16 21:45:05.250136 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:05.250057    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" path="/var/lib/kubelet/pods/ae7c8c38-4b98-41b7-81dc-8add79435b63/volumes"
Jul 16 21:45:14.243858 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:14.243830    2572 scope.go:117] "RemoveContainer" containerID="5c8ab253e489c6670322489bf205725a760cdeef66c02e5f2e77c223e4a38435"
Jul 16 21:45:14.870517 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:14.870483    2572 generic.go:356] "Generic (PLEG): container finished" podID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerID="f33341476eb72da17b664efa0aaed744761250c4a6568718455a5a0d1aa1488c" exitCode=7
Jul 16 21:45:14.870660 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:14.870550    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" event={"ID":"9815a822-5f4d-4302-a6cf-b54f8a0a22cb","Type":"ContainerDied","Data":"f33341476eb72da17b664efa0aaed744761250c4a6568718455a5a0d1aa1488c"}
Jul 16 21:45:14.870660 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:14.870612    2572 scope.go:117] "RemoveContainer" containerID="5c8ab253e489c6670322489bf205725a760cdeef66c02e5f2e77c223e4a38435"
Jul 16 21:45:14.870862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:14.870842    2572 scope.go:117] "RemoveContainer" containerID="f33341476eb72da17b664efa0aaed744761250c4a6568718455a5a0d1aa1488c"
Jul 16 21:45:14.871111 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:45:14.871093    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9_odh-ai-gateway-infra(9815a822-5f4d-4302-a6cf-b54f8a0a22cb)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb"
Jul 16 21:45:15.940644 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:15.940606    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"]
Jul 16 21:45:16.078110 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:16.078076    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"
Jul 16 21:45:16.884430 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:16.884400    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"
Jul 16 21:45:16.884430 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:16.884412    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9" event={"ID":"9815a822-5f4d-4302-a6cf-b54f8a0a22cb","Type":"ContainerDied","Data":"ad762d1ec6726bc76deb67b00fec00c39a074636c59cbd93f1cb2c32576b9041"}
Jul 16 21:45:16.884662 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:16.884473    2572 scope.go:117] "RemoveContainer" containerID="f33341476eb72da17b664efa0aaed744761250c4a6568718455a5a0d1aa1488c"
Jul 16 21:45:16.932415 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:16.932370    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"]
Jul 16 21:45:16.938798 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:16.938770    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-fa7d409d-29737305-zzvq9"]
Jul 16 21:45:17.249699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:17.249615    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" path="/var/lib/kubelet/pods/9815a822-5f4d-4302-a6cf-b54f8a0a22cb/volumes"
Jul 16 21:45:44.643111 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643069    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"]
Jul 16 21:45:44.643763 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643743    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:45:44.643808 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643768    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:45:44.643808 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643783    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.643808 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643791    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.643930 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643813    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:45:44.643930 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643823    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:45:44.643930 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643839    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.643930 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643848    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.643930 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643870    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.643930 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643878    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.644111 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643975    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:45:44.644111 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643987    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.644111 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.643998    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:45:44.644111 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.644007    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.644111 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.644017    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae7c8c38-4b98-41b7-81dc-8add79435b63" containerName="revoke-keys"
Jul 16 21:45:44.648607 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.648585    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"
Jul 16 21:45:44.652022 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.651989    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:45:44.653584 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.653439    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"]
Jul 16 21:45:44.692678 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.692641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6dac555-bc3e-4598-ad8a-e73810146d54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg\" (UID: \"b6dac555-bc3e-4598-ad8a-e73810146d54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"
Jul 16 21:45:44.793980 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.793945    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6dac555-bc3e-4598-ad8a-e73810146d54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg\" (UID: \"b6dac555-bc3e-4598-ad8a-e73810146d54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"
Jul 16 21:45:44.794603 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.794578    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6dac555-bc3e-4598-ad8a-e73810146d54-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg\" (UID: \"b6dac555-bc3e-4598-ad8a-e73810146d54\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"
Jul 16 21:45:44.961315 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:44.961227    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"
Jul 16 21:45:45.109012 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:45.108982    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"]
Jul 16 21:45:45.110686 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:45:45.110658    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6dac555_bc3e_4598_ad8a_e73810146d54.slice/crio-824bacee122c67ad52eb4de98b3c99da39ffa0f179c2ccd86bde25498a71b8b5 WatchSource:0}: Error finding container 824bacee122c67ad52eb4de98b3c99da39ffa0f179c2ccd86bde25498a71b8b5: Status 404 returned error can't find the container with id 824bacee122c67ad52eb4de98b3c99da39ffa0f179c2ccd86bde25498a71b8b5
Jul 16 21:45:46.045316 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:46.045279    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerID="14cd30ae052d3e96bd1b7bafbba3421ef561beebf3fc1f2449b44a13e8d42398" exitCode=7
Jul 16 21:45:46.045316 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:46.045318    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg" event={"ID":"b6dac555-bc3e-4598-ad8a-e73810146d54","Type":"ContainerDied","Data":"14cd30ae052d3e96bd1b7bafbba3421ef561beebf3fc1f2449b44a13e8d42398"}
Jul 16 21:45:46.045870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:46.045389    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg" event={"ID":"b6dac555-bc3e-4598-ad8a-e73810146d54","Type":"ContainerStarted","Data":"824bacee122c67ad52eb4de98b3c99da39ffa0f179c2ccd86bde25498a71b8b5"}
Jul 16 21:45:46.045870 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:46.045623    2572 scope.go:117] "RemoveContainer" containerID="14cd30ae052d3e96bd1b7bafbba3421ef561beebf3fc1f2449b44a13e8d42398"
Jul 16 21:45:47.052472 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:47.052430    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerID="c6a9b581a0c1b3a121fac3c84d4272fce03d9a7cdea6868b4ce89348c7c3b186" exitCode=7
Jul 16 21:45:47.052985 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:47.052520    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg" event={"ID":"b6dac555-bc3e-4598-ad8a-e73810146d54","Type":"ContainerDied","Data":"c6a9b581a0c1b3a121fac3c84d4272fce03d9a7cdea6868b4ce89348c7c3b186"}
Jul 16 21:45:47.052985 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:47.052583    2572 scope.go:117] "RemoveContainer" containerID="14cd30ae052d3e96bd1b7bafbba3421ef561beebf3fc1f2449b44a13e8d42398"
Jul 16 21:45:47.052985 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:47.052826    2572 scope.go:117] "RemoveContainer" containerID="c6a9b581a0c1b3a121fac3c84d4272fce03d9a7cdea6868b4ce89348c7c3b186"
Jul 16 21:45:47.053105 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:45:47.053048    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg_odh-ai-gateway-infra(b6dac555-bc3e-4598-ad8a-e73810146d54)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54"
Jul 16 21:45:48.059722 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:48.059697    2572 scope.go:117] "RemoveContainer" containerID="c6a9b581a0c1b3a121fac3c84d4272fce03d9a7cdea6868b4ce89348c7c3b186"
Jul 16 21:45:48.060108 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:45:48.059875    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg_odh-ai-gateway-infra(b6dac555-bc3e-4598-ad8a-e73810146d54)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54"
Jul 16 21:45:54.565587 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:54.565553    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"]
Jul 16 21:45:54.566084 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:54.565763    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy" containerID="cri-o://adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423" gracePeriod=30
Jul 16 21:45:55.592798 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:55.592755    2572 patch_prober.go:28] interesting pod/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:45:55.593168 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:55.592823    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:45:59.244422 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.244347    2572 scope.go:117] "RemoveContainer" containerID="c6a9b581a0c1b3a121fac3c84d4272fce03d9a7cdea6868b4ce89348c7c3b186"
Jul 16 21:45:59.878158 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.878128    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:45:59.936609 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936581    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-podinfo\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.936766 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936620    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-data\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.936766 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936659    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-envoy\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.936766 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936673    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-certs\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.936766 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936694    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-token\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.936766 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936716    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-credential-socket\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.936766 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936732    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istiod-ca-cert\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.937087 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936804    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpdmd\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-kube-api-access-dpdmd\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.937087 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.936859    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-socket\") pod \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\" (UID: \"e903ea0c-511a-49be-aa9d-aabe4e0601b6\") "
Jul 16 21:45:59.937197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.937067    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:45:59.937197 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.937184    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-certs\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:45:59.937325 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.937304    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:45:59.937533 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.937487    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:45:59.937667 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.937552    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:45:59.937789 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.937766    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:45:59.939551 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.939522    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:45:59.939551 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.939539    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-kube-api-access-dpdmd" (OuterVolumeSpecName: "kube-api-access-dpdmd") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "kube-api-access-dpdmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:45:59.939718 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.939699    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:45:59.939718 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:45:59.939704    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "e903ea0c-511a-49be-aa9d-aabe4e0601b6" (UID: "e903ea0c-511a-49be-aa9d-aabe4e0601b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:46:00.038195 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.038160    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-credential-socket\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.038195 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.038191    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istiod-ca-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.038195 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.038201    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpdmd\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-kube-api-access-dpdmd\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.038490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.038210    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-workload-socket\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.038490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.038219    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-podinfo\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.038490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.038227    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-data\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.038490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.038234    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-envoy\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.038490 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.038241    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e903ea0c-511a-49be-aa9d-aabe4e0601b6-istio-token\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:00.130494 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.130459    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerID="97f2206aa5c9740f6333c22a1fcf98d28ea18d00d9ea7b2e6cc9f19d9244d74a" exitCode=7
Jul 16 21:46:00.130686 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.130537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg" event={"ID":"b6dac555-bc3e-4598-ad8a-e73810146d54","Type":"ContainerDied","Data":"97f2206aa5c9740f6333c22a1fcf98d28ea18d00d9ea7b2e6cc9f19d9244d74a"}
Jul 16 21:46:00.130686 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.130601    2572 scope.go:117] "RemoveContainer" containerID="c6a9b581a0c1b3a121fac3c84d4272fce03d9a7cdea6868b4ce89348c7c3b186"
Jul 16 21:46:00.130915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.130896    2572 scope.go:117] "RemoveContainer" containerID="97f2206aa5c9740f6333c22a1fcf98d28ea18d00d9ea7b2e6cc9f19d9244d74a"
Jul 16 21:46:00.131153 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:46:00.131132    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg_odh-ai-gateway-infra(b6dac555-bc3e-4598-ad8a-e73810146d54)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54"
Jul 16 21:46:00.135215 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.132775    2572 generic.go:356] "Generic (PLEG): container finished" podID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerID="adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423" exitCode=0
Jul 16 21:46:00.135215 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.132897    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" event={"ID":"e903ea0c-511a-49be-aa9d-aabe4e0601b6","Type":"ContainerDied","Data":"adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423"}
Jul 16 21:46:00.135215 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.132940    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk" event={"ID":"e903ea0c-511a-49be-aa9d-aabe4e0601b6","Type":"ContainerDied","Data":"5a0fb6edba60cbd5dffd874594c96e3be2d3472d92b6b1a45e814991f78632d9"}
Jul 16 21:46:00.135215 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.133010    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"
Jul 16 21:46:00.151196 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.151162    2572 scope.go:117] "RemoveContainer" containerID="adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423"
Jul 16 21:46:00.161000 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.160981    2572 scope.go:117] "RemoveContainer" containerID="f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2"
Jul 16 21:46:00.170316 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.170282    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"]
Jul 16 21:46:00.171458 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.171441    2572 scope.go:117] "RemoveContainer" containerID="adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423"
Jul 16 21:46:00.171768 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:46:00.171746    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423\": container with ID starting with adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423 not found: ID does not exist" containerID="adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423"
Jul 16 21:46:00.171844 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.171779    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423"} err="failed to get container status \"adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423\": rpc error: code = NotFound desc = could not find container \"adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423\": container with ID starting with adc29fec51c968f9c2fb2cf321c3cb701081d3714a1a3b2711bcfd827959d423 not found: ID does not exist"
Jul 16 21:46:00.171844 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.171798    2572 scope.go:117] "RemoveContainer" containerID="f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2"
Jul 16 21:46:00.172086 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:46:00.172064    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2\": container with ID starting with f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2 not found: ID does not exist" containerID="f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2"
Jul 16 21:46:00.172163 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.172095    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2"} err="failed to get container status \"f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2\": rpc error: code = NotFound desc = could not find container \"f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2\": container with ID starting with f811e1dbe1ec8169843f87b9a3b6c50eb164c1e7f7710326ff35d5273c30f1f2 not found: ID does not exist"
Jul 16 21:46:00.175472 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:00.175450    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fa7d409d-openshift-default-59cbb468df-m7lsk"]
Jul 16 21:46:01.170705 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:01.170670    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"]
Jul 16 21:46:01.251062 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:01.251030    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" path="/var/lib/kubelet/pods/e903ea0c-511a-49be-aa9d-aabe4e0601b6/volumes"
Jul 16 21:46:01.314603 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:01.314573    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"
Jul 16 21:46:01.449935 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:01.449843    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6dac555-bc3e-4598-ad8a-e73810146d54-maas-api-service-ca\") pod \"b6dac555-bc3e-4598-ad8a-e73810146d54\" (UID: \"b6dac555-bc3e-4598-ad8a-e73810146d54\") "
Jul 16 21:46:01.450255 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:01.450233    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6dac555-bc3e-4598-ad8a-e73810146d54-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b6dac555-bc3e-4598-ad8a-e73810146d54" (UID: "b6dac555-bc3e-4598-ad8a-e73810146d54"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:46:01.550624 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:01.550589    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6dac555-bc3e-4598-ad8a-e73810146d54-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:02.147464 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:02.147425    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg" event={"ID":"b6dac555-bc3e-4598-ad8a-e73810146d54","Type":"ContainerDied","Data":"824bacee122c67ad52eb4de98b3c99da39ffa0f179c2ccd86bde25498a71b8b5"}
Jul 16 21:46:02.147644 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:02.147476    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"
Jul 16 21:46:02.147644 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:02.147490    2572 scope.go:117] "RemoveContainer" containerID="97f2206aa5c9740f6333c22a1fcf98d28ea18d00d9ea7b2e6cc9f19d9244d74a"
Jul 16 21:46:02.189408 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:02.189368    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"]
Jul 16 21:46:02.205563 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:02.205531    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-tvdpg"]
Jul 16 21:46:03.249762 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:03.249726    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" path="/var/lib/kubelet/pods/b6dac555-bc3e-4598-ad8a-e73810146d54/volumes"
Jul 16 21:46:08.466014 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.465975    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"]
Jul 16 21:46:08.466717 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466694    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy"
Jul 16 21:46:08.466775 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466721    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy"
Jul 16 21:46:08.466775 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466741    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:08.466775 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466752    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:08.466775 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466767    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:08.466900 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466777    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:08.466900 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466790    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:46:08.466900 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466798    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:46:08.466987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466924    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:08.466987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466938    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy"
Jul 16 21:46:08.466987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466947    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy"
Jul 16 21:46:08.466987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466961    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9815a822-5f4d-4302-a6cf-b54f8a0a22cb" containerName="cleanup"
Jul 16 21:46:08.466987 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.466971    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:08.470638 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.470611    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.474262 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.474236    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-7c0a35cd-gw-openshift-default-dockercfg-pw2dv\""
Jul 16 21:46:08.488514 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.488479    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"]
Jul 16 21:46:08.621460 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621419    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-podinfo\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.621617 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621470    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-token\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.621617 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621598    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-envoy\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.621718 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621636    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-socket\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.621718 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621673    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-credential-socket\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.621718 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621706    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz27r\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-kube-api-access-tz27r\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.621873 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a72911-f3bf-4d69-ba46-0df27de110cb-istiod-ca-cert\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.621873 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621837    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-data\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.621970 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.621877    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-certs\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723398 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723272    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-envoy\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723398 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723314    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-socket\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723398 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723336    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-credential-socket\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723398 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723389    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz27r\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-kube-api-access-tz27r\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723751 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723419    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a72911-f3bf-4d69-ba46-0df27de110cb-istiod-ca-cert\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723751 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723440    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-data\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723751 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723457    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-certs\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723751 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723479    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-podinfo\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.723751 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723505    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-token\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.724029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723794    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-socket\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.724029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.723930    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-data\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.724029 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.724012    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-certs\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.724157 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.724137    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-credential-socket\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.724200 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.724152    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a72911-f3bf-4d69-ba46-0df27de110cb-istiod-ca-cert\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.725738 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.725719    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-envoy\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.725954 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.725936    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-podinfo\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.732617 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.732588    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-token\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.732747 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.732731    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz27r\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-kube-api-access-tz27r\") pod \"e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") " pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.782531 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.782498    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:08.927186 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.927152    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"]
Jul 16 21:46:08.928258 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:46:08.928231    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a72911_f3bf_4d69_ba46_0df27de110cb.slice/crio-f563ca9aebb9b9efd3672c8ef947dcde8a52896a37f4d6a1abbe5e9fbba7d78b WatchSource:0}: Error finding container f563ca9aebb9b9efd3672c8ef947dcde8a52896a37f4d6a1abbe5e9fbba7d78b: Status 404 returned error can't find the container with id f563ca9aebb9b9efd3672c8ef947dcde8a52896a37f4d6a1abbe5e9fbba7d78b
Jul 16 21:46:08.930751 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.930720    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:46:08.930817 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.930789    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:46:08.930859 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:08.930821    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:46:09.192579 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:09.192536    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" event={"ID":"37a72911-f3bf-4d69-ba46-0df27de110cb","Type":"ContainerStarted","Data":"f6b4cd0d9365023129cecc5b8db55530a059cb6a859140d1928362361a07b640"}
Jul 16 21:46:09.192747 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:09.192609    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" event={"ID":"37a72911-f3bf-4d69-ba46-0df27de110cb","Type":"ContainerStarted","Data":"f563ca9aebb9b9efd3672c8ef947dcde8a52896a37f4d6a1abbe5e9fbba7d78b"}
Jul 16 21:46:09.215386 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:09.215288    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" podStartSLOduration=1.215270441 podStartE2EDuration="1.215270441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:46:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:46:09.21413725 +0000 UTC m=+1834.725296996" watchObservedRunningTime="2026-07-16 21:46:09.215270441 +0000 UTC m=+1834.726430176"
Jul 16 21:46:09.782715 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:09.782678    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:09.789495 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:09.789466    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:10.200058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:10.199975    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:10.201603 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:10.201577    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:20.258023 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:20.257934    2572 generic.go:356] "Generic (PLEG): container finished" podID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerID="f6b4cd0d9365023129cecc5b8db55530a059cb6a859140d1928362361a07b640" exitCode=0
Jul 16 21:46:20.258023 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:20.258008    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" event={"ID":"37a72911-f3bf-4d69-ba46-0df27de110cb","Type":"ContainerDied","Data":"f6b4cd0d9365023129cecc5b8db55530a059cb6a859140d1928362361a07b640"}
Jul 16 21:46:20.258526 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:20.258493    2572 scope.go:117] "RemoveContainer" containerID="f6b4cd0d9365023129cecc5b8db55530a059cb6a859140d1928362361a07b640"
Jul 16 21:46:20.259553 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:20.259522    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:46:20.259635 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:20.259595    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:46:20.259674 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:20.259631    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:46:21.265449 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:21.265412    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" event={"ID":"37a72911-f3bf-4d69-ba46-0df27de110cb","Type":"ContainerStarted","Data":"8cd6dbce7c5bd9396b5f64916180b63db5d6ea1f871408e3f70b6c6cb02c8bc1"}
Jul 16 21:46:21.783040 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:21.782998    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:21.784370 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:21.784310    2572 patch_prober.go:28] interesting pod/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 16 21:46:21.784646 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:21.784618    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 16 21:46:22.783462 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:22.783424    2572 patch_prober.go:28] interesting pod/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 16 21:46:22.784125 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:22.784091    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 16 21:46:23.784027 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:23.783975    2572 patch_prober.go:28] interesting pod/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 16 21:46:23.784594 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:23.784062    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 16 21:46:24.783388 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:24.783333    2572 patch_prober.go:28] interesting pod/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 16 21:46:24.783604 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:24.783426    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 16 21:46:25.200417 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:25.200291    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:25.786721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:25.786692    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:25.787699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:25.787680    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:46:34.701878 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.701842    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"]
Jul 16 21:46:34.702288 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.702274    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy"
Jul 16 21:46:34.702288 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.702289    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e903ea0c-511a-49be-aa9d-aabe4e0601b6" containerName="istio-proxy"
Jul 16 21:46:34.702402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.702302    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:34.702402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.702308    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:34.702402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.702394    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6dac555-bc3e-4598-ad8a-e73810146d54" containerName="revoke-keys"
Jul 16 21:46:34.705852 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.705834    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"
Jul 16 21:46:34.708610 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.708590    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:46:34.714285 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.714260    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"]
Jul 16 21:46:34.798332 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.798294    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fe8ebe-6583-4595-b3c7-2901a14ab236-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz\" (UID: \"d9fe8ebe-6583-4595-b3c7-2901a14ab236\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"
Jul 16 21:46:34.899715 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.899676    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fe8ebe-6583-4595-b3c7-2901a14ab236-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz\" (UID: \"d9fe8ebe-6583-4595-b3c7-2901a14ab236\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"
Jul 16 21:46:34.900311 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:34.900291    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fe8ebe-6583-4595-b3c7-2901a14ab236-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz\" (UID: \"d9fe8ebe-6583-4595-b3c7-2901a14ab236\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"
Jul 16 21:46:35.017940 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:35.017821    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"
Jul 16 21:46:35.160139 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:35.160111    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"]
Jul 16 21:46:35.161882 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:46:35.161850    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9fe8ebe_6583_4595_b3c7_2901a14ab236.slice/crio-6a2ff066560cecff3e296432acbe45fde3c030dea5e8e4e0409d8ac7eb29b3a9 WatchSource:0}: Error finding container 6a2ff066560cecff3e296432acbe45fde3c030dea5e8e4e0409d8ac7eb29b3a9: Status 404 returned error can't find the container with id 6a2ff066560cecff3e296432acbe45fde3c030dea5e8e4e0409d8ac7eb29b3a9
Jul 16 21:46:35.352993 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:35.352957    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" event={"ID":"d9fe8ebe-6583-4595-b3c7-2901a14ab236","Type":"ContainerStarted","Data":"6a2ff066560cecff3e296432acbe45fde3c030dea5e8e4e0409d8ac7eb29b3a9"}
Jul 16 21:46:36.360155 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:36.360116    2572 generic.go:356] "Generic (PLEG): container finished" podID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerID="c8edb9f130fefac9f8ba7a450ed2ba15b2635bfd87a031683303b5a76708d14e" exitCode=6
Jul 16 21:46:36.360656 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:36.360197    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" event={"ID":"d9fe8ebe-6583-4595-b3c7-2901a14ab236","Type":"ContainerDied","Data":"c8edb9f130fefac9f8ba7a450ed2ba15b2635bfd87a031683303b5a76708d14e"}
Jul 16 21:46:36.360656 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:36.360508    2572 scope.go:117] "RemoveContainer" containerID="c8edb9f130fefac9f8ba7a450ed2ba15b2635bfd87a031683303b5a76708d14e"
Jul 16 21:46:37.367585 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:37.367550    2572 generic.go:356] "Generic (PLEG): container finished" podID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerID="108d0bafaa37bc3ba137c9e5b0bd778f9dabf934277fff9d3749fb512a83c9dc" exitCode=6
Jul 16 21:46:37.368003 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:37.367633    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" event={"ID":"d9fe8ebe-6583-4595-b3c7-2901a14ab236","Type":"ContainerDied","Data":"108d0bafaa37bc3ba137c9e5b0bd778f9dabf934277fff9d3749fb512a83c9dc"}
Jul 16 21:46:37.368003 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:37.367687    2572 scope.go:117] "RemoveContainer" containerID="c8edb9f130fefac9f8ba7a450ed2ba15b2635bfd87a031683303b5a76708d14e"
Jul 16 21:46:37.368003 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:37.367954    2572 scope.go:117] "RemoveContainer" containerID="108d0bafaa37bc3ba137c9e5b0bd778f9dabf934277fff9d3749fb512a83c9dc"
Jul 16 21:46:37.368210 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:46:37.368188    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz_odh-ai-gateway-infra(d9fe8ebe-6583-4595-b3c7-2901a14ab236)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236"
Jul 16 21:46:38.375726 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:38.375698    2572 scope.go:117] "RemoveContainer" containerID="108d0bafaa37bc3ba137c9e5b0bd778f9dabf934277fff9d3749fb512a83c9dc"
Jul 16 21:46:38.376121 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:46:38.375936    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz_odh-ai-gateway-infra(d9fe8ebe-6583-4595-b3c7-2901a14ab236)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236"
Jul 16 21:46:52.243721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:52.243686    2572 scope.go:117] "RemoveContainer" containerID="108d0bafaa37bc3ba137c9e5b0bd778f9dabf934277fff9d3749fb512a83c9dc"
Jul 16 21:46:52.452011 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:52.451978    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" event={"ID":"d9fe8ebe-6583-4595-b3c7-2901a14ab236","Type":"ContainerStarted","Data":"e5d7055d18a78a7b41e2ec2e92d7c5c3694e99153e8256f9daca6a5ac2836c63"}
Jul 16 21:46:52.472235 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:52.472170    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" podStartSLOduration=18.472154473 podStartE2EDuration="18.472154473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:46:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:46:52.470558903 +0000 UTC m=+1877.981718639" watchObservedRunningTime="2026-07-16 21:46:52.472154473 +0000 UTC m=+1877.983314207"
Jul 16 21:46:53.459009 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:53.458965    2572 generic.go:356] "Generic (PLEG): container finished" podID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerID="e5d7055d18a78a7b41e2ec2e92d7c5c3694e99153e8256f9daca6a5ac2836c63" exitCode=6
Jul 16 21:46:53.459483 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:53.459037    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" event={"ID":"d9fe8ebe-6583-4595-b3c7-2901a14ab236","Type":"ContainerDied","Data":"e5d7055d18a78a7b41e2ec2e92d7c5c3694e99153e8256f9daca6a5ac2836c63"}
Jul 16 21:46:53.459483 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:53.459100    2572 scope.go:117] "RemoveContainer" containerID="108d0bafaa37bc3ba137c9e5b0bd778f9dabf934277fff9d3749fb512a83c9dc"
Jul 16 21:46:53.459483 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:53.459346    2572 scope.go:117] "RemoveContainer" containerID="e5d7055d18a78a7b41e2ec2e92d7c5c3694e99153e8256f9daca6a5ac2836c63"
Jul 16 21:46:53.459644 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:46:53.459621    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz_odh-ai-gateway-infra(d9fe8ebe-6583-4595-b3c7-2901a14ab236)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236"
Jul 16 21:46:53.486701 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:53.486660    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"]
Jul 16 21:46:54.608194 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:54.608159    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"
Jul 16 21:46:54.688960 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:54.688921    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fe8ebe-6583-4595-b3c7-2901a14ab236-maas-api-service-ca\") pod \"d9fe8ebe-6583-4595-b3c7-2901a14ab236\" (UID: \"d9fe8ebe-6583-4595-b3c7-2901a14ab236\") "
Jul 16 21:46:54.689254 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:54.689229    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9fe8ebe-6583-4595-b3c7-2901a14ab236-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9fe8ebe-6583-4595-b3c7-2901a14ab236" (UID: "d9fe8ebe-6583-4595-b3c7-2901a14ab236"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:46:54.790589 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:54.790553    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9fe8ebe-6583-4595-b3c7-2901a14ab236-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:46:55.473760 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:55.473729    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"
Jul 16 21:46:55.473760 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:55.473752    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz" event={"ID":"d9fe8ebe-6583-4595-b3c7-2901a14ab236","Type":"ContainerDied","Data":"6a2ff066560cecff3e296432acbe45fde3c030dea5e8e4e0409d8ac7eb29b3a9"}
Jul 16 21:46:55.474002 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:55.473819    2572 scope.go:117] "RemoveContainer" containerID="e5d7055d18a78a7b41e2ec2e92d7c5c3694e99153e8256f9daca6a5ac2836c63"
Jul 16 21:46:55.491904 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:55.491871    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"]
Jul 16 21:46:55.497643 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:55.497612    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-dmnnz"]
Jul 16 21:46:57.249539 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:46:57.249461    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" path="/var/lib/kubelet/pods/d9fe8ebe-6583-4595-b3c7-2901a14ab236/volumes"
Jul 16 21:47:14.308676 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.308630    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"]
Jul 16 21:47:14.309159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.309041    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:14.309159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.309052    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:14.309159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.309066    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:14.309159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.309071    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:14.309159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.309133    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:14.309159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.309144    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:14.309159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.309150    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:14.312158 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.312133    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"
Jul 16 21:47:14.314990 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.314964    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:47:14.321997 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.321960    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"]
Jul 16 21:47:14.359578 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.359539    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"]
Jul 16 21:47:14.359863 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.359826    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy" containerID="cri-o://8cd6dbce7c5bd9396b5f64916180b63db5d6ea1f871408e3f70b6c6cb02c8bc1" gracePeriod=30
Jul 16 21:47:14.382269 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.382229    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84e0df1-745e-4b9b-9eef-0fae344e1f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6\" (UID: \"a84e0df1-745e-4b9b-9eef-0fae344e1f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"
Jul 16 21:47:14.483385 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.483318    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84e0df1-745e-4b9b-9eef-0fae344e1f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6\" (UID: \"a84e0df1-745e-4b9b-9eef-0fae344e1f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"
Jul 16 21:47:14.484254 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.484222    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84e0df1-745e-4b9b-9eef-0fae344e1f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6\" (UID: \"a84e0df1-745e-4b9b-9eef-0fae344e1f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"
Jul 16 21:47:14.625228 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.625135    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"
Jul 16 21:47:14.769659 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:14.769632    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"]
Jul 16 21:47:14.771550 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:47:14.771519    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda84e0df1_745e_4b9b_9eef_0fae344e1f9f.slice/crio-28f329f05a890a62d2fc8ae60f7efe95398cd96349f80d1c7897c1ecfdd190c7 WatchSource:0}: Error finding container 28f329f05a890a62d2fc8ae60f7efe95398cd96349f80d1c7897c1ecfdd190c7: Status 404 returned error can't find the container with id 28f329f05a890a62d2fc8ae60f7efe95398cd96349f80d1c7897c1ecfdd190c7
Jul 16 21:47:15.589771 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:15.589738    2572 generic.go:356] "Generic (PLEG): container finished" podID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerID="1442ed6be3e44dd5d5dfc97120b1d08a205fb667b0d8d5819008de47d8ae5287" exitCode=7
Jul 16 21:47:15.590184 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:15.589790    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" event={"ID":"a84e0df1-745e-4b9b-9eef-0fae344e1f9f","Type":"ContainerDied","Data":"1442ed6be3e44dd5d5dfc97120b1d08a205fb667b0d8d5819008de47d8ae5287"}
Jul 16 21:47:15.590184 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:15.589857    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" event={"ID":"a84e0df1-745e-4b9b-9eef-0fae344e1f9f","Type":"ContainerStarted","Data":"28f329f05a890a62d2fc8ae60f7efe95398cd96349f80d1c7897c1ecfdd190c7"}
Jul 16 21:47:15.590184 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:15.590058    2572 scope.go:117] "RemoveContainer" containerID="1442ed6be3e44dd5d5dfc97120b1d08a205fb667b0d8d5819008de47d8ae5287"
Jul 16 21:47:16.597144 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:16.597048    2572 generic.go:356] "Generic (PLEG): container finished" podID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerID="1fa4073e4966505dd5172878aa6a76221304ab79afb54ae9875965f19cd87b55" exitCode=7
Jul 16 21:47:16.597626 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:16.597136    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" event={"ID":"a84e0df1-745e-4b9b-9eef-0fae344e1f9f","Type":"ContainerDied","Data":"1fa4073e4966505dd5172878aa6a76221304ab79afb54ae9875965f19cd87b55"}
Jul 16 21:47:16.597626 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:16.597207    2572 scope.go:117] "RemoveContainer" containerID="1442ed6be3e44dd5d5dfc97120b1d08a205fb667b0d8d5819008de47d8ae5287"
Jul 16 21:47:16.597626 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:16.597432    2572 scope.go:117] "RemoveContainer" containerID="1fa4073e4966505dd5172878aa6a76221304ab79afb54ae9875965f19cd87b55"
Jul 16 21:47:16.597795 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:47:16.597662    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6_odh-ai-gateway-infra(a84e0df1-745e-4b9b-9eef-0fae344e1f9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f"
Jul 16 21:47:17.604799 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:17.604771    2572 scope.go:117] "RemoveContainer" containerID="1fa4073e4966505dd5172878aa6a76221304ab79afb54ae9875965f19cd87b55"
Jul 16 21:47:17.605197 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:47:17.604987    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6_odh-ai-gateway-infra(a84e0df1-745e-4b9b-9eef-0fae344e1f9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f"
Jul 16 21:47:19.619628 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.619547    2572 generic.go:356] "Generic (PLEG): container finished" podID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerID="8cd6dbce7c5bd9396b5f64916180b63db5d6ea1f871408e3f70b6c6cb02c8bc1" exitCode=0
Jul 16 21:47:19.619628 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.619617    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" event={"ID":"37a72911-f3bf-4d69-ba46-0df27de110cb","Type":"ContainerDied","Data":"8cd6dbce7c5bd9396b5f64916180b63db5d6ea1f871408e3f70b6c6cb02c8bc1"}
Jul 16 21:47:19.620004 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.619676    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg" event={"ID":"37a72911-f3bf-4d69-ba46-0df27de110cb","Type":"ContainerDied","Data":"f563ca9aebb9b9efd3672c8ef947dcde8a52896a37f4d6a1abbe5e9fbba7d78b"}
Jul 16 21:47:19.620004 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.619699    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f563ca9aebb9b9efd3672c8ef947dcde8a52896a37f4d6a1abbe5e9fbba7d78b"
Jul 16 21:47:19.620004 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.619716    2572 scope.go:117] "RemoveContainer" containerID="f6b4cd0d9365023129cecc5b8db55530a059cb6a859140d1928362361a07b640"
Jul 16 21:47:19.626526 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.626497    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:47:19.736418 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736329    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-socket\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736418 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736422    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a72911-f3bf-4d69-ba46-0df27de110cb-istiod-ca-cert\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736685 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736448    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-podinfo\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736685 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736466    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-data\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736685 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736495    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz27r\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-kube-api-access-tz27r\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736685 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736622    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:47:19.736685 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736652    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-envoy\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736967 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736711    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-certs\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736967 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736770    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-credential-socket\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736967 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736801    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-token\") pod \"37a72911-f3bf-4d69-ba46-0df27de110cb\" (UID: \"37a72911-f3bf-4d69-ba46-0df27de110cb\") "
Jul 16 21:47:19.736967 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.736851    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37a72911-f3bf-4d69-ba46-0df27de110cb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:47:19.737176 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.737061    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:47:19.737176 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.737133    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:47:19.737176 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.737162    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-socket\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:19.737326 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.737180    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37a72911-f3bf-4d69-ba46-0df27de110cb-istiod-ca-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:19.737326 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.737194    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-workload-certs\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:19.737326 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.737246    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-data" (OuterVolumeSpecName: "istio-data") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:47:19.738833 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.738802    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:47:19.738833 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.738816    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-kube-api-access-tz27r" (OuterVolumeSpecName: "kube-api-access-tz27r") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "kube-api-access-tz27r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:47:19.738833 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.738826    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:47:19.739289 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.739256    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-token" (OuterVolumeSpecName: "istio-token") pod "37a72911-f3bf-4d69-ba46-0df27de110cb" (UID: "37a72911-f3bf-4d69-ba46-0df27de110cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:47:19.837814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.837758    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-podinfo\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:19.837814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.837808    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-data\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:19.837814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.837820    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz27r\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-kube-api-access-tz27r\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:19.837814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.837831    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-envoy\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:19.838097 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.837840    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37a72911-f3bf-4d69-ba46-0df27de110cb-credential-socket\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:19.838097 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:19.837847    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37a72911-f3bf-4d69-ba46-0df27de110cb-istio-token\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:20.626375 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:20.626329    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"
Jul 16 21:47:20.664122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:20.664085    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"]
Jul 16 21:47:20.670613 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:20.670585    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-7c0a35cd-gw-openshift-default-5cd9785f59-qn4wg"]
Jul 16 21:47:21.250416 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:21.250296    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" path="/var/lib/kubelet/pods/37a72911-f3bf-4d69-ba46-0df27de110cb/volumes"
Jul 16 21:47:26.961143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.961105    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"]
Jul 16 21:47:26.961857 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.961833    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy"
Jul 16 21:47:26.961953 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.961861    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy"
Jul 16 21:47:26.961953 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.961898    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy"
Jul 16 21:47:26.961953 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.961907    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy"
Jul 16 21:47:26.961953 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.961921    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:26.961953 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.961930    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9fe8ebe-6583-4595-b3c7-2901a14ab236" containerName="revoke-keys"
Jul 16 21:47:26.962194 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.962025    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy"
Jul 16 21:47:26.966835 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.966810    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:26.969293 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.969266    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f6d0b514-openshift-default-dockercfg-2d4c8\""
Jul 16 21:47:26.979179 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:26.979148    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"]
Jul 16 21:47:27.109076 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109035    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-certs\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.109268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109131    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-data\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.109268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109170    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istiod-ca-cert\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.109268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109234    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-token\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.109426 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109281    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-envoy\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.109426 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-socket\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.109426 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109324    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-podinfo\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.109426 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109403    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-credential-socket\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.109426 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.109421    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc96h\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-kube-api-access-bc96h\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210153 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210109    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-envoy\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210153 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210155    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-socket\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210433 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210173    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-podinfo\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210433 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210223    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-credential-socket\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210433 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210241    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc96h\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-kube-api-access-bc96h\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210433 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210264    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-certs\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210433 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210317    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-data\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210702 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210488    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istiod-ca-cert\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210702 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210559    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-token\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210702 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210648    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-socket\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210702 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210683    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-data\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210775    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-certs\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.210941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.210852    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-credential-socket\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.211238 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.211183    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istiod-ca-cert\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.212541 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.212515    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-envoy\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.212810 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.212792    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-podinfo\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.225087 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.225028    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc96h\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-kube-api-access-bc96h\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.225087 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.225088    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-token\") pod \"e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") " pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.279583 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.279543    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:27.427420 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.427391    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"]
Jul 16 21:47:27.429245 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:47:27.429219    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92dcdb0f_9eac_4bfb_95e4_76bcb3327e05.slice/crio-fdac65129169a804487b66f4c41f113b1ca7e948a83640e8bc12b83cb0825492 WatchSource:0}: Error finding container fdac65129169a804487b66f4c41f113b1ca7e948a83640e8bc12b83cb0825492: Status 404 returned error can't find the container with id fdac65129169a804487b66f4c41f113b1ca7e948a83640e8bc12b83cb0825492
Jul 16 21:47:27.431480 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.431441    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:47:27.431841 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.431511    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:47:27.431841 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.431539    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:47:27.666157 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.666114    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" event={"ID":"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05","Type":"ContainerStarted","Data":"1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46"}
Jul 16 21:47:27.666393 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.666186    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" event={"ID":"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05","Type":"ContainerStarted","Data":"fdac65129169a804487b66f4c41f113b1ca7e948a83640e8bc12b83cb0825492"}
Jul 16 21:47:27.692721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:27.692664    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" podStartSLOduration=1.692644765 podStartE2EDuration="1.692644765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:27.69128555 +0000 UTC m=+1913.202445275" watchObservedRunningTime="2026-07-16 21:47:27.692644765 +0000 UTC m=+1913.203804498"
Jul 16 21:47:28.280919 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:28.280849    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:28.286078 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:28.286053    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:28.671874 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:28.671791    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:28.672695 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:28.672677    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:33.244720 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:33.244687    2572 scope.go:117] "RemoveContainer" containerID="1fa4073e4966505dd5172878aa6a76221304ab79afb54ae9875965f19cd87b55"
Jul 16 21:47:33.714577 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:33.714537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" event={"ID":"a84e0df1-745e-4b9b-9eef-0fae344e1f9f","Type":"ContainerStarted","Data":"dd03f73ccd59621e95baf37f2005bd27a25dd036b1e5bc668d8079b69fa59a3e"}
Jul 16 21:47:33.732391 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:33.732315    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" podStartSLOduration=19.732299743 podStartE2EDuration="19.732299743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:47:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:47:33.730281443 +0000 UTC m=+1919.241441181" watchObservedRunningTime="2026-07-16 21:47:33.732299743 +0000 UTC m=+1919.243459473"
Jul 16 21:47:34.722220 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:34.722186    2572 generic.go:356] "Generic (PLEG): container finished" podID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerID="dd03f73ccd59621e95baf37f2005bd27a25dd036b1e5bc668d8079b69fa59a3e" exitCode=6
Jul 16 21:47:34.722692 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:34.722254    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" event={"ID":"a84e0df1-745e-4b9b-9eef-0fae344e1f9f","Type":"ContainerDied","Data":"dd03f73ccd59621e95baf37f2005bd27a25dd036b1e5bc668d8079b69fa59a3e"}
Jul 16 21:47:34.722692 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:34.722320    2572 scope.go:117] "RemoveContainer" containerID="1fa4073e4966505dd5172878aa6a76221304ab79afb54ae9875965f19cd87b55"
Jul 16 21:47:34.722787 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:34.722706    2572 scope.go:117] "RemoveContainer" containerID="dd03f73ccd59621e95baf37f2005bd27a25dd036b1e5bc668d8079b69fa59a3e"
Jul 16 21:47:34.722981 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:47:34.722957    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6_odh-ai-gateway-infra(a84e0df1-745e-4b9b-9eef-0fae344e1f9f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f"
Jul 16 21:47:34.741248 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:34.741217    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"]
Jul 16 21:47:35.903202 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:35.903169    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"
Jul 16 21:47:36.002721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:36.002683    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84e0df1-745e-4b9b-9eef-0fae344e1f9f-maas-api-service-ca\") pod \"a84e0df1-745e-4b9b-9eef-0fae344e1f9f\" (UID: \"a84e0df1-745e-4b9b-9eef-0fae344e1f9f\") "
Jul 16 21:47:36.003085 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:36.003060    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a84e0df1-745e-4b9b-9eef-0fae344e1f9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a84e0df1-745e-4b9b-9eef-0fae344e1f9f" (UID: "a84e0df1-745e-4b9b-9eef-0fae344e1f9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:47:36.104152 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:36.104110    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a84e0df1-745e-4b9b-9eef-0fae344e1f9f-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:47:36.736069 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:36.736039    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"
Jul 16 21:47:36.736253 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:36.736068    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6" event={"ID":"a84e0df1-745e-4b9b-9eef-0fae344e1f9f","Type":"ContainerDied","Data":"28f329f05a890a62d2fc8ae60f7efe95398cd96349f80d1c7897c1ecfdd190c7"}
Jul 16 21:47:36.736253 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:36.736142    2572 scope.go:117] "RemoveContainer" containerID="dd03f73ccd59621e95baf37f2005bd27a25dd036b1e5bc668d8079b69fa59a3e"
Jul 16 21:47:36.764254 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:36.764212    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"]
Jul 16 21:47:36.770302 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:36.770273    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-f4bc6"]
Jul 16 21:47:37.250541 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:37.250512    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" path="/var/lib/kubelet/pods/a84e0df1-745e-4b9b-9eef-0fae344e1f9f/volumes"
Jul 16 21:47:38.752284 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:38.752195    2572 generic.go:356] "Generic (PLEG): container finished" podID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerID="1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46" exitCode=0
Jul 16 21:47:38.752284 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:38.752266    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" event={"ID":"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05","Type":"ContainerDied","Data":"1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46"}
Jul 16 21:47:38.752768 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:38.752745    2572 scope.go:117] "RemoveContainer" containerID="1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46"
Jul 16 21:47:38.753790 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:38.753763    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:47:38.753855 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:38.753844    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:47:38.753894 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:38.753886    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 16 21:47:39.759891 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:39.759858    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" event={"ID":"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05","Type":"ContainerStarted","Data":"7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8"}
Jul 16 21:47:40.280066 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:40.280034    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:40.281464 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:40.281424    2572 patch_prober.go:28] interesting pod/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 16 21:47:40.281600 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:40.281480    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 16 21:47:41.280595 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:41.280539    2572 patch_prober.go:28] interesting pod/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 16 21:47:41.280995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:41.280612    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 16 21:47:42.280474 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:42.280434    2572 patch_prober.go:28] interesting pod/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 16 21:47:42.280653 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:42.280496    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 16 21:47:43.280585 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:43.280542    2572 patch_prober.go:28] interesting pod/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 16 21:47:43.280776 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:43.280605    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 16 21:47:43.672140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:43.672049    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:44.284658 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:44.284623    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:47:44.285612 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:47:44.285586    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:48:14.365143 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365105    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"]
Jul 16 21:48:14.365734 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365711    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:14.365814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365766    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:14.365814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365804    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:14.365814 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365813    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:14.365961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365942    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:14.365961 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365954    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:14.366061 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365963    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:14.366061 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.365976    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="37a72911-f3bf-4d69-ba46-0df27de110cb" containerName="istio-proxy"
Jul 16 21:48:14.368583 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.368561    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"
Jul 16 21:48:14.371283 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.371265    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:48:14.377530 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.377505    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"]
Jul 16 21:48:14.462243 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.462198    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60069672-8dfb-4384-9623-b4043640dc46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp\" (UID: \"60069672-8dfb-4384-9623-b4043640dc46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"
Jul 16 21:48:14.563007 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.562964    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60069672-8dfb-4384-9623-b4043640dc46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp\" (UID: \"60069672-8dfb-4384-9623-b4043640dc46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"
Jul 16 21:48:14.563643 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.563623    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60069672-8dfb-4384-9623-b4043640dc46-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp\" (UID: \"60069672-8dfb-4384-9623-b4043640dc46\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"
Jul 16 21:48:14.680414 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.680270    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"
Jul 16 21:48:14.818038 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.818005    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"]
Jul 16 21:48:14.819284 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:48:14.819261    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60069672_8dfb_4384_9623_b4043640dc46.slice/crio-2cfb0cb564f6836b05026c49f110fa6e56b857a375c627ac2d1a4dfef5fada05 WatchSource:0}: Error finding container 2cfb0cb564f6836b05026c49f110fa6e56b857a375c627ac2d1a4dfef5fada05: Status 404 returned error can't find the container with id 2cfb0cb564f6836b05026c49f110fa6e56b857a375c627ac2d1a4dfef5fada05
Jul 16 21:48:14.966294 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:14.966208    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp" event={"ID":"60069672-8dfb-4384-9623-b4043640dc46","Type":"ContainerStarted","Data":"2cfb0cb564f6836b05026c49f110fa6e56b857a375c627ac2d1a4dfef5fada05"}
Jul 16 21:48:15.973511 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:15.973479    2572 generic.go:356] "Generic (PLEG): container finished" podID="60069672-8dfb-4384-9623-b4043640dc46" containerID="205cc79682079dd548dd8553d73959244b9502302aaf44b70d225d1e5f2273c6" exitCode=6
Jul 16 21:48:15.973916 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:15.973569    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp" event={"ID":"60069672-8dfb-4384-9623-b4043640dc46","Type":"ContainerDied","Data":"205cc79682079dd548dd8553d73959244b9502302aaf44b70d225d1e5f2273c6"}
Jul 16 21:48:15.973916 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:15.973734    2572 scope.go:117] "RemoveContainer" containerID="205cc79682079dd548dd8553d73959244b9502302aaf44b70d225d1e5f2273c6"
Jul 16 21:48:16.987832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:16.987741    2572 generic.go:356] "Generic (PLEG): container finished" podID="60069672-8dfb-4384-9623-b4043640dc46" containerID="3a17766bd5c88c4f179bd7ed8e334b14fecf166120bb8d889e95f9ad3b7ef949" exitCode=6
Jul 16 21:48:16.987832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:16.987786    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp" event={"ID":"60069672-8dfb-4384-9623-b4043640dc46","Type":"ContainerDied","Data":"3a17766bd5c88c4f179bd7ed8e334b14fecf166120bb8d889e95f9ad3b7ef949"}
Jul 16 21:48:16.988330 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:16.987867    2572 scope.go:117] "RemoveContainer" containerID="205cc79682079dd548dd8553d73959244b9502302aaf44b70d225d1e5f2273c6"
Jul 16 21:48:16.988330 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:16.988070    2572 scope.go:117] "RemoveContainer" containerID="3a17766bd5c88c4f179bd7ed8e334b14fecf166120bb8d889e95f9ad3b7ef949"
Jul 16 21:48:16.988330 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:48:16.988269    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp_odh-ai-gateway-infra(60069672-8dfb-4384-9623-b4043640dc46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp" podUID="60069672-8dfb-4384-9623-b4043640dc46"
Jul 16 21:48:17.995047 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:17.995017    2572 scope.go:117] "RemoveContainer" containerID="3a17766bd5c88c4f179bd7ed8e334b14fecf166120bb8d889e95f9ad3b7ef949"
Jul 16 21:48:17.995471 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:48:17.995194    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp_odh-ai-gateway-infra(60069672-8dfb-4384-9623-b4043640dc46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp" podUID="60069672-8dfb-4384-9623-b4043640dc46"
Jul 16 21:48:31.245046 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:31.244964    2572 scope.go:117] "RemoveContainer" containerID="3a17766bd5c88c4f179bd7ed8e334b14fecf166120bb8d889e95f9ad3b7ef949"
Jul 16 21:48:32.081121 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:32.081037    2572 generic.go:356] "Generic (PLEG): container finished" podID="60069672-8dfb-4384-9623-b4043640dc46" containerID="4a82731196f44a60370f63ef311648364d6e0dbb17dbe20747605e9b29bad233" exitCode=6
Jul 16 21:48:32.081121 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:32.081111    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp" event={"ID":"60069672-8dfb-4384-9623-b4043640dc46","Type":"ContainerDied","Data":"4a82731196f44a60370f63ef311648364d6e0dbb17dbe20747605e9b29bad233"}
Jul 16 21:48:32.081320 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:32.081171    2572 scope.go:117] "RemoveContainer" containerID="3a17766bd5c88c4f179bd7ed8e334b14fecf166120bb8d889e95f9ad3b7ef949"
Jul 16 21:48:32.081481 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:32.081460    2572 scope.go:117] "RemoveContainer" containerID="4a82731196f44a60370f63ef311648364d6e0dbb17dbe20747605e9b29bad233"
Jul 16 21:48:32.081706 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:48:32.081684    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp_odh-ai-gateway-infra(60069672-8dfb-4384-9623-b4043640dc46)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp" podUID="60069672-8dfb-4384-9623-b4043640dc46"
Jul 16 21:48:33.115759 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:33.115726    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"]
Jul 16 21:48:33.259581 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:33.259553    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"
Jul 16 21:48:33.345956 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:33.345924    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60069672-8dfb-4384-9623-b4043640dc46-maas-api-service-ca\") pod \"60069672-8dfb-4384-9623-b4043640dc46\" (UID: \"60069672-8dfb-4384-9623-b4043640dc46\") "
Jul 16 21:48:33.346304 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:33.346269    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60069672-8dfb-4384-9623-b4043640dc46-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "60069672-8dfb-4384-9623-b4043640dc46" (UID: "60069672-8dfb-4384-9623-b4043640dc46"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:48:33.446899 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:33.446804    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60069672-8dfb-4384-9623-b4043640dc46-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:48:34.095413 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.095385    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"
Jul 16 21:48:34.095602 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.095405    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp" event={"ID":"60069672-8dfb-4384-9623-b4043640dc46","Type":"ContainerDied","Data":"2cfb0cb564f6836b05026c49f110fa6e56b857a375c627ac2d1a4dfef5fada05"}
Jul 16 21:48:34.095602 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.095507    2572 scope.go:117] "RemoveContainer" containerID="4a82731196f44a60370f63ef311648364d6e0dbb17dbe20747605e9b29bad233"
Jul 16 21:48:34.118651 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.118621    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"]
Jul 16 21:48:34.130208 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.130173    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-krjbp"]
Jul 16 21:48:34.802275 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802237    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"]
Jul 16 21:48:34.802804 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802784    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:34.802862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802808    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a84e0df1-745e-4b9b-9eef-0fae344e1f9f" containerName="revoke-keys"
Jul 16 21:48:34.802862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802837    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.802862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802844    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.802862 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802856    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.802990 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802864    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.802990 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802886    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.802990 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802892    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.802990 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802986    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.803102 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.802996    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.803102 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.803003    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="60069672-8dfb-4384-9623-b4043640dc46" containerName="revoke-keys"
Jul 16 21:48:34.806328 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.806305    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"
Jul 16 21:48:34.808859 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.808837    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:48:34.813686 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.813661    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"]
Jul 16 21:48:34.960167 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:34.960126    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t\" (UID: \"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"
Jul 16 21:48:35.061741 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:35.061642    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t\" (UID: \"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"
Jul 16 21:48:35.062277 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:35.062256    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t\" (UID: \"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"
Jul 16 21:48:35.122101 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:35.122071    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:48:35.130249 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:35.130222    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"
Jul 16 21:48:35.253564 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:35.253533    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60069672-8dfb-4384-9623-b4043640dc46" path="/var/lib/kubelet/pods/60069672-8dfb-4384-9623-b4043640dc46/volumes"
Jul 16 21:48:35.271843 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:35.271813    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"]
Jul 16 21:48:35.272872 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:48:35.272841    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b49e2a_4dcd_4ccc_ab09_1a2bcafe8f48.slice/crio-32418894bd19eb72e10a1f73727e84ee42d5fe195d5bc4b2afa86d4b0caeaa0e WatchSource:0}: Error finding container 32418894bd19eb72e10a1f73727e84ee42d5fe195d5bc4b2afa86d4b0caeaa0e: Status 404 returned error can't find the container with id 32418894bd19eb72e10a1f73727e84ee42d5fe195d5bc4b2afa86d4b0caeaa0e
Jul 16 21:48:36.109787 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:36.109749    2572 generic.go:356] "Generic (PLEG): container finished" podID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerID="455a0e0a68ea27931daba56fe2ac3c10752ffb05811965a386d90f287017754f" exitCode=6
Jul 16 21:48:36.109965 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:36.109841    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t" event={"ID":"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48","Type":"ContainerDied","Data":"455a0e0a68ea27931daba56fe2ac3c10752ffb05811965a386d90f287017754f"}
Jul 16 21:48:36.109965 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:36.109894    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t" event={"ID":"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48","Type":"ContainerStarted","Data":"32418894bd19eb72e10a1f73727e84ee42d5fe195d5bc4b2afa86d4b0caeaa0e"}
Jul 16 21:48:36.110050 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:36.110036    2572 scope.go:117] "RemoveContainer" containerID="455a0e0a68ea27931daba56fe2ac3c10752ffb05811965a386d90f287017754f"
Jul 16 21:48:37.116876 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:37.116835    2572 generic.go:356] "Generic (PLEG): container finished" podID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerID="63fe237a219a913ae5b645d896ede48853fb03f42a1846ff4da9248935551fb8" exitCode=6
Jul 16 21:48:37.117402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:37.116901    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t" event={"ID":"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48","Type":"ContainerDied","Data":"63fe237a219a913ae5b645d896ede48853fb03f42a1846ff4da9248935551fb8"}
Jul 16 21:48:37.117402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:37.116954    2572 scope.go:117] "RemoveContainer" containerID="455a0e0a68ea27931daba56fe2ac3c10752ffb05811965a386d90f287017754f"
Jul 16 21:48:37.117402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:37.117141    2572 scope.go:117] "RemoveContainer" containerID="63fe237a219a913ae5b645d896ede48853fb03f42a1846ff4da9248935551fb8"
Jul 16 21:48:37.117549 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:48:37.117425    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t_odh-ai-gateway-infra(e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48"
Jul 16 21:48:38.124535 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:38.124506    2572 scope.go:117] "RemoveContainer" containerID="63fe237a219a913ae5b645d896ede48853fb03f42a1846ff4da9248935551fb8"
Jul 16 21:48:38.124925 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:48:38.124719    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t_odh-ai-gateway-infra(e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48"
Jul 16 21:48:51.243766 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:51.243729    2572 scope.go:117] "RemoveContainer" containerID="63fe237a219a913ae5b645d896ede48853fb03f42a1846ff4da9248935551fb8"
Jul 16 21:48:52.208777 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:52.208692    2572 generic.go:356] "Generic (PLEG): container finished" podID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerID="5ab9283e1bdb20b9e946240c4fc2cd2d1d067f2bb1ed173b0ea0936b95f00ab5" exitCode=6
Jul 16 21:48:52.208777 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:52.208764    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t" event={"ID":"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48","Type":"ContainerDied","Data":"5ab9283e1bdb20b9e946240c4fc2cd2d1d067f2bb1ed173b0ea0936b95f00ab5"}
Jul 16 21:48:52.208954 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:52.208857    2572 scope.go:117] "RemoveContainer" containerID="63fe237a219a913ae5b645d896ede48853fb03f42a1846ff4da9248935551fb8"
Jul 16 21:48:52.209264 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:52.209247    2572 scope.go:117] "RemoveContainer" containerID="5ab9283e1bdb20b9e946240c4fc2cd2d1d067f2bb1ed173b0ea0936b95f00ab5"
Jul 16 21:48:52.209525 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:48:52.209505    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t_odh-ai-gateway-infra(e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48"
Jul 16 21:48:53.237251 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:53.237214    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"]
Jul 16 21:48:53.380068 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:53.380036    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"
Jul 16 21:48:53.435467 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:53.435433    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48-maas-api-service-ca\") pod \"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48\" (UID: \"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48\") "
Jul 16 21:48:53.435812 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:53.435790    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" (UID: "e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:48:53.536778 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:53.536746    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:48:54.223941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:54.223906    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t" event={"ID":"e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48","Type":"ContainerDied","Data":"32418894bd19eb72e10a1f73727e84ee42d5fe195d5bc4b2afa86d4b0caeaa0e"}
Jul 16 21:48:54.223941 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:54.223937    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"
Jul 16 21:48:54.224222 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:54.223971    2572 scope.go:117] "RemoveContainer" containerID="5ab9283e1bdb20b9e946240c4fc2cd2d1d067f2bb1ed173b0ea0936b95f00ab5"
Jul 16 21:48:54.249001 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:54.248969    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"]
Jul 16 21:48:54.264246 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:54.264217    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-twd6t"]
Jul 16 21:48:55.249676 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:48:55.249631    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" path="/var/lib/kubelet/pods/e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48/volumes"
Jul 16 21:49:33.300281 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300242    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"]
Jul 16 21:49:33.300821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300684    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:33.300821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300698    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:33.300821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300715    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:33.300821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300720    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:33.300821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300729    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:33.300821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300735    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:33.300821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300789    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:33.300821 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.300797    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:33.304026 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.304007    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"
Jul 16 21:49:33.307484 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.307463    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:49:33.314228 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.314200    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"]
Jul 16 21:49:33.421819 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.421775    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37fd380b-472c-4ff5-a266-9f98ed0c9752-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8\" (UID: \"37fd380b-472c-4ff5-a266-9f98ed0c9752\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"
Jul 16 21:49:33.522451 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.522415    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37fd380b-472c-4ff5-a266-9f98ed0c9752-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8\" (UID: \"37fd380b-472c-4ff5-a266-9f98ed0c9752\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"
Jul 16 21:49:33.523101 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.523077    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37fd380b-472c-4ff5-a266-9f98ed0c9752-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8\" (UID: \"37fd380b-472c-4ff5-a266-9f98ed0c9752\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"
Jul 16 21:49:33.616132 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.616039    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"
Jul 16 21:49:33.758837 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:33.758806    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"]
Jul 16 21:49:33.760620 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:49:33.760590    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37fd380b_472c_4ff5_a266_9f98ed0c9752.slice/crio-2414a15855f9fe0b6503f245c1b2cf59bc2e2e2464e3e74e9b948477f1548584 WatchSource:0}: Error finding container 2414a15855f9fe0b6503f245c1b2cf59bc2e2e2464e3e74e9b948477f1548584: Status 404 returned error can't find the container with id 2414a15855f9fe0b6503f245c1b2cf59bc2e2e2464e3e74e9b948477f1548584
Jul 16 21:49:34.457535 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:34.457495    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" event={"ID":"37fd380b-472c-4ff5-a266-9f98ed0c9752","Type":"ContainerStarted","Data":"00909a455d2a651d37891fb4dd5ff6d01a8ad9d51448ec9306ed7f94dce490ec"}
Jul 16 21:49:34.457931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:34.457560    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" event={"ID":"37fd380b-472c-4ff5-a266-9f98ed0c9752","Type":"ContainerStarted","Data":"2414a15855f9fe0b6503f245c1b2cf59bc2e2e2464e3e74e9b948477f1548584"}
Jul 16 21:49:34.475396 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:34.475323    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" podStartSLOduration=1.475305761 podStartE2EDuration="1.475305761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:34.473399144 +0000 UTC m=+2039.984558879" watchObservedRunningTime="2026-07-16 21:49:34.475305761 +0000 UTC m=+2039.986465558"
Jul 16 21:49:35.464272 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:35.464236    2572 generic.go:356] "Generic (PLEG): container finished" podID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerID="00909a455d2a651d37891fb4dd5ff6d01a8ad9d51448ec9306ed7f94dce490ec" exitCode=7
Jul 16 21:49:35.464808 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:35.464326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" event={"ID":"37fd380b-472c-4ff5-a266-9f98ed0c9752","Type":"ContainerDied","Data":"00909a455d2a651d37891fb4dd5ff6d01a8ad9d51448ec9306ed7f94dce490ec"}
Jul 16 21:49:35.464808 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:35.464624    2572 scope.go:117] "RemoveContainer" containerID="00909a455d2a651d37891fb4dd5ff6d01a8ad9d51448ec9306ed7f94dce490ec"
Jul 16 21:49:36.472645 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:36.472552    2572 generic.go:356] "Generic (PLEG): container finished" podID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerID="3f5788c0dc4220e0ae4434bb57d41cd902959e441fcbdb64302ac96eeb015594" exitCode=7
Jul 16 21:49:36.473043 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:36.472642    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" event={"ID":"37fd380b-472c-4ff5-a266-9f98ed0c9752","Type":"ContainerDied","Data":"3f5788c0dc4220e0ae4434bb57d41cd902959e441fcbdb64302ac96eeb015594"}
Jul 16 21:49:36.473043 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:36.472702    2572 scope.go:117] "RemoveContainer" containerID="00909a455d2a651d37891fb4dd5ff6d01a8ad9d51448ec9306ed7f94dce490ec"
Jul 16 21:49:36.473043 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:36.472844    2572 scope.go:117] "RemoveContainer" containerID="3f5788c0dc4220e0ae4434bb57d41cd902959e441fcbdb64302ac96eeb015594"
Jul 16 21:49:36.473148 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:49:36.473069    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8_odh-ai-gateway-infra(37fd380b-472c-4ff5-a266-9f98ed0c9752)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752"
Jul 16 21:49:37.479883 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:37.479853    2572 scope.go:117] "RemoveContainer" containerID="3f5788c0dc4220e0ae4434bb57d41cd902959e441fcbdb64302ac96eeb015594"
Jul 16 21:49:37.480283 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:49:37.480037    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8_odh-ai-gateway-infra(37fd380b-472c-4ff5-a266-9f98ed0c9752)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752"
Jul 16 21:49:51.244104 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:51.244070    2572 scope.go:117] "RemoveContainer" containerID="3f5788c0dc4220e0ae4434bb57d41cd902959e441fcbdb64302ac96eeb015594"
Jul 16 21:49:51.561427 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:51.561392    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" event={"ID":"37fd380b-472c-4ff5-a266-9f98ed0c9752","Type":"ContainerStarted","Data":"1a1ad9c700c5d9037e9e8e9601c7d86f2fed372a20cc0b57979afbdf1180b77f"}
Jul 16 21:49:52.569052 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:52.569019    2572 generic.go:356] "Generic (PLEG): container finished" podID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerID="1a1ad9c700c5d9037e9e8e9601c7d86f2fed372a20cc0b57979afbdf1180b77f" exitCode=7
Jul 16 21:49:52.569515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:52.569101    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" event={"ID":"37fd380b-472c-4ff5-a266-9f98ed0c9752","Type":"ContainerDied","Data":"1a1ad9c700c5d9037e9e8e9601c7d86f2fed372a20cc0b57979afbdf1180b77f"}
Jul 16 21:49:52.569515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:52.569169    2572 scope.go:117] "RemoveContainer" containerID="3f5788c0dc4220e0ae4434bb57d41cd902959e441fcbdb64302ac96eeb015594"
Jul 16 21:49:52.569515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:52.569462    2572 scope.go:117] "RemoveContainer" containerID="1a1ad9c700c5d9037e9e8e9601c7d86f2fed372a20cc0b57979afbdf1180b77f"
Jul 16 21:49:52.569729 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:49:52.569704    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8_odh-ai-gateway-infra(37fd380b-472c-4ff5-a266-9f98ed0c9752)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752"
Jul 16 21:49:52.597932 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:52.597896    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"]
Jul 16 21:49:53.713673 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:53.713642    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"
Jul 16 21:49:53.822978 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:53.822940    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37fd380b-472c-4ff5-a266-9f98ed0c9752-maas-api-service-ca\") pod \"37fd380b-472c-4ff5-a266-9f98ed0c9752\" (UID: \"37fd380b-472c-4ff5-a266-9f98ed0c9752\") "
Jul 16 21:49:53.823311 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:53.823285    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37fd380b-472c-4ff5-a266-9f98ed0c9752-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37fd380b-472c-4ff5-a266-9f98ed0c9752" (UID: "37fd380b-472c-4ff5-a266-9f98ed0c9752"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:49:53.924053 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:53.923957    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37fd380b-472c-4ff5-a266-9f98ed0c9752-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:49:54.450725 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.450687    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"]
Jul 16 21:49:54.451094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451081    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:49:54.451140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451095    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:49:54.451140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451104    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:49:54.451140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451110    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:49:54.451140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451124    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:49:54.451140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451130    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:49:54.451298 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451189    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8b49e2a-4dcd-4ccc-ab09-1a2bcafe8f48" containerName="revoke-keys"
Jul 16 21:49:54.451298 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451197    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:49:54.451298 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.451204    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:49:54.454182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.454162    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"
Jul 16 21:49:54.462012 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.461975    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"]
Jul 16 21:49:54.583314 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.583281    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"
Jul 16 21:49:54.583527 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.583326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8" event={"ID":"37fd380b-472c-4ff5-a266-9f98ed0c9752","Type":"ContainerDied","Data":"2414a15855f9fe0b6503f245c1b2cf59bc2e2e2464e3e74e9b948477f1548584"}
Jul 16 21:49:54.583527 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.583410    2572 scope.go:117] "RemoveContainer" containerID="1a1ad9c700c5d9037e9e8e9601c7d86f2fed372a20cc0b57979afbdf1180b77f"
Jul 16 21:49:54.609483 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.609452    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"]
Jul 16 21:49:54.613649 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.613623    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-gh5k8"]
Jul 16 21:49:54.630686 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.630658    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb84fd1-4dd3-41be-9619-8ad113dc8d24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s\" (UID: \"9fb84fd1-4dd3-41be-9619-8ad113dc8d24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"
Jul 16 21:49:54.732028 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.731923    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb84fd1-4dd3-41be-9619-8ad113dc8d24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s\" (UID: \"9fb84fd1-4dd3-41be-9619-8ad113dc8d24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"
Jul 16 21:49:54.732661 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.732641    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb84fd1-4dd3-41be-9619-8ad113dc8d24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s\" (UID: \"9fb84fd1-4dd3-41be-9619-8ad113dc8d24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"
Jul 16 21:49:54.766327 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.766292    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"
Jul 16 21:49:54.906516 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:54.906490    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"]
Jul 16 21:49:54.908132 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:49:54.908110    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fb84fd1_4dd3_41be_9619_8ad113dc8d24.slice/crio-c613c836597a66b82f3914cafa3afcb8f3f7994835fc072827f89fb2d9d6bd8d WatchSource:0}: Error finding container c613c836597a66b82f3914cafa3afcb8f3f7994835fc072827f89fb2d9d6bd8d: Status 404 returned error can't find the container with id c613c836597a66b82f3914cafa3afcb8f3f7994835fc072827f89fb2d9d6bd8d
Jul 16 21:49:55.250715 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:55.250681    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" path="/var/lib/kubelet/pods/37fd380b-472c-4ff5-a266-9f98ed0c9752/volumes"
Jul 16 21:49:55.591995 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:55.591941    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" event={"ID":"9fb84fd1-4dd3-41be-9619-8ad113dc8d24","Type":"ContainerStarted","Data":"b55a157d5baff2a4e0d75a0fa33ea097ddb978e01690d6e833051e3f62064c48"}
Jul 16 21:49:55.592067 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:55.592004    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" event={"ID":"9fb84fd1-4dd3-41be-9619-8ad113dc8d24","Type":"ContainerStarted","Data":"c613c836597a66b82f3914cafa3afcb8f3f7994835fc072827f89fb2d9d6bd8d"}
Jul 16 21:49:55.611011 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:55.610927    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" podStartSLOduration=1.6109094929999999 podStartE2EDuration="1.610909493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:49:55.609509603 +0000 UTC m=+2061.120669337" watchObservedRunningTime="2026-07-16 21:49:55.610909493 +0000 UTC m=+2061.122069226"
Jul 16 21:49:56.599009 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:56.598923    2572 generic.go:356] "Generic (PLEG): container finished" podID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerID="b55a157d5baff2a4e0d75a0fa33ea097ddb978e01690d6e833051e3f62064c48" exitCode=6
Jul 16 21:49:56.599402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:56.599018    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" event={"ID":"9fb84fd1-4dd3-41be-9619-8ad113dc8d24","Type":"ContainerDied","Data":"b55a157d5baff2a4e0d75a0fa33ea097ddb978e01690d6e833051e3f62064c48"}
Jul 16 21:49:56.599402 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:56.599186    2572 scope.go:117] "RemoveContainer" containerID="b55a157d5baff2a4e0d75a0fa33ea097ddb978e01690d6e833051e3f62064c48"
Jul 16 21:49:57.606489 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:57.606453    2572 generic.go:356] "Generic (PLEG): container finished" podID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerID="2b3f6d6efb208809608f22aa4ca15a5a7965df2993ab2d669c27012486150045" exitCode=6
Jul 16 21:49:57.606958 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:57.606544    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" event={"ID":"9fb84fd1-4dd3-41be-9619-8ad113dc8d24","Type":"ContainerDied","Data":"2b3f6d6efb208809608f22aa4ca15a5a7965df2993ab2d669c27012486150045"}
Jul 16 21:49:57.606958 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:57.606628    2572 scope.go:117] "RemoveContainer" containerID="b55a157d5baff2a4e0d75a0fa33ea097ddb978e01690d6e833051e3f62064c48"
Jul 16 21:49:57.606958 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:57.606907    2572 scope.go:117] "RemoveContainer" containerID="2b3f6d6efb208809608f22aa4ca15a5a7965df2993ab2d669c27012486150045"
Jul 16 21:49:57.607141 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:49:57.607119    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s_odh-ai-gateway-infra(9fb84fd1-4dd3-41be-9619-8ad113dc8d24)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24"
Jul 16 21:49:58.613633 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:49:58.613603    2572 scope.go:117] "RemoveContainer" containerID="2b3f6d6efb208809608f22aa4ca15a5a7965df2993ab2d669c27012486150045"
Jul 16 21:49:58.614108 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:49:58.613872    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s_odh-ai-gateway-infra(9fb84fd1-4dd3-41be-9619-8ad113dc8d24)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24"
Jul 16 21:50:14.244260 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:14.244225    2572 scope.go:117] "RemoveContainer" containerID="2b3f6d6efb208809608f22aa4ca15a5a7965df2993ab2d669c27012486150045"
Jul 16 21:50:14.708692 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:14.708659    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" event={"ID":"9fb84fd1-4dd3-41be-9619-8ad113dc8d24","Type":"ContainerStarted","Data":"25f3453cee2ac1c7e40505a81aa1a506885247e3d43a3b9a1d3041ce9fbca45a"}
Jul 16 21:50:15.716120 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:15.716086    2572 generic.go:356] "Generic (PLEG): container finished" podID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerID="25f3453cee2ac1c7e40505a81aa1a506885247e3d43a3b9a1d3041ce9fbca45a" exitCode=6
Jul 16 21:50:15.716588 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:15.716161    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" event={"ID":"9fb84fd1-4dd3-41be-9619-8ad113dc8d24","Type":"ContainerDied","Data":"25f3453cee2ac1c7e40505a81aa1a506885247e3d43a3b9a1d3041ce9fbca45a"}
Jul 16 21:50:15.716588 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:15.716220    2572 scope.go:117] "RemoveContainer" containerID="2b3f6d6efb208809608f22aa4ca15a5a7965df2993ab2d669c27012486150045"
Jul 16 21:50:15.716588 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:15.716524    2572 scope.go:117] "RemoveContainer" containerID="25f3453cee2ac1c7e40505a81aa1a506885247e3d43a3b9a1d3041ce9fbca45a"
Jul 16 21:50:15.716750 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:50:15.716728    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s_odh-ai-gateway-infra(9fb84fd1-4dd3-41be-9619-8ad113dc8d24)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24"
Jul 16 21:50:15.760115 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:15.754928    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"]
Jul 16 21:50:16.867993 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:16.867963    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"
Jul 16 21:50:16.941136 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:16.941088    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb84fd1-4dd3-41be-9619-8ad113dc8d24-maas-api-service-ca\") pod \"9fb84fd1-4dd3-41be-9619-8ad113dc8d24\" (UID: \"9fb84fd1-4dd3-41be-9619-8ad113dc8d24\") "
Jul 16 21:50:16.941491 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:16.941467    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fb84fd1-4dd3-41be-9619-8ad113dc8d24-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9fb84fd1-4dd3-41be-9619-8ad113dc8d24" (UID: "9fb84fd1-4dd3-41be-9619-8ad113dc8d24"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:50:17.042532 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:17.042497    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fb84fd1-4dd3-41be-9619-8ad113dc8d24-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:50:17.731035 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:17.731004    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"
Jul 16 21:50:17.731246 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:17.731005    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s" event={"ID":"9fb84fd1-4dd3-41be-9619-8ad113dc8d24","Type":"ContainerDied","Data":"c613c836597a66b82f3914cafa3afcb8f3f7994835fc072827f89fb2d9d6bd8d"}
Jul 16 21:50:17.731246 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:17.731155    2572 scope.go:117] "RemoveContainer" containerID="25f3453cee2ac1c7e40505a81aa1a506885247e3d43a3b9a1d3041ce9fbca45a"
Jul 16 21:50:17.748531 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:17.748503    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"]
Jul 16 21:50:17.752983 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:17.752956    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-ps22s"]
Jul 16 21:50:19.250478 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:19.250444    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" path="/var/lib/kubelet/pods/9fb84fd1-4dd3-41be-9619-8ad113dc8d24/volumes"
Jul 16 21:50:33.368426 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.368387    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"]
Jul 16 21:50:33.368923 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.368906    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:33.368968 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.368926    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:33.368968 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.368962    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:33.369030 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.368970    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:33.369068 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.369046    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="37fd380b-472c-4ff5-a266-9f98ed0c9752" containerName="revoke-keys"
Jul 16 21:50:33.369068 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.369057    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:33.369068 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.369063    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:33.369157 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.369073    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:33.372306 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.372286    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"
Jul 16 21:50:33.374975 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.374952    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:50:33.383046 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.383017    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"]
Jul 16 21:50:33.506177 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.506142    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faba5e83-7968-4013-a13f-5939d8a4bd70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh\" (UID: \"faba5e83-7968-4013-a13f-5939d8a4bd70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"
Jul 16 21:50:33.606721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.606682    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faba5e83-7968-4013-a13f-5939d8a4bd70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh\" (UID: \"faba5e83-7968-4013-a13f-5939d8a4bd70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"
Jul 16 21:50:33.607317 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.607296    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faba5e83-7968-4013-a13f-5939d8a4bd70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh\" (UID: \"faba5e83-7968-4013-a13f-5939d8a4bd70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"
Jul 16 21:50:33.686159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.686073    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"
Jul 16 21:50:33.829746 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:33.829710    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"]
Jul 16 21:50:33.831398 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:50:33.831347    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaba5e83_7968_4013_a13f_5939d8a4bd70.slice/crio-e30ed8be08f7166e49561602da5f71b28e37a05d5324df0145505b638dc44430 WatchSource:0}: Error finding container e30ed8be08f7166e49561602da5f71b28e37a05d5324df0145505b638dc44430: Status 404 returned error can't find the container with id e30ed8be08f7166e49561602da5f71b28e37a05d5324df0145505b638dc44430
Jul 16 21:50:34.828544 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:34.828497    2572 generic.go:356] "Generic (PLEG): container finished" podID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerID="e2c2adeb708b5ca55d9ca8692af06ec9ab82040e1e7f96972d203eb0a8585698" exitCode=7
Jul 16 21:50:34.828983 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:34.828588    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh" event={"ID":"faba5e83-7968-4013-a13f-5939d8a4bd70","Type":"ContainerDied","Data":"e2c2adeb708b5ca55d9ca8692af06ec9ab82040e1e7f96972d203eb0a8585698"}
Jul 16 21:50:34.828983 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:34.828647    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh" event={"ID":"faba5e83-7968-4013-a13f-5939d8a4bd70","Type":"ContainerStarted","Data":"e30ed8be08f7166e49561602da5f71b28e37a05d5324df0145505b638dc44430"}
Jul 16 21:50:34.828983 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:34.828825    2572 scope.go:117] "RemoveContainer" containerID="e2c2adeb708b5ca55d9ca8692af06ec9ab82040e1e7f96972d203eb0a8585698"
Jul 16 21:50:35.836063 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:35.836029    2572 generic.go:356] "Generic (PLEG): container finished" podID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerID="d35392993c12af4847df512caefca64decaf1bca83b5cc931ee3570ba0835491" exitCode=7
Jul 16 21:50:35.836572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:35.836105    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh" event={"ID":"faba5e83-7968-4013-a13f-5939d8a4bd70","Type":"ContainerDied","Data":"d35392993c12af4847df512caefca64decaf1bca83b5cc931ee3570ba0835491"}
Jul 16 21:50:35.836572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:35.836159    2572 scope.go:117] "RemoveContainer" containerID="e2c2adeb708b5ca55d9ca8692af06ec9ab82040e1e7f96972d203eb0a8585698"
Jul 16 21:50:35.836572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:35.836415    2572 scope.go:117] "RemoveContainer" containerID="d35392993c12af4847df512caefca64decaf1bca83b5cc931ee3570ba0835491"
Jul 16 21:50:35.836691 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:50:35.836648    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh_odh-ai-gateway-infra(faba5e83-7968-4013-a13f-5939d8a4bd70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70"
Jul 16 21:50:36.843262 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:36.843232    2572 scope.go:117] "RemoveContainer" containerID="d35392993c12af4847df512caefca64decaf1bca83b5cc931ee3570ba0835491"
Jul 16 21:50:36.843674 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:50:36.843427    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh_odh-ai-gateway-infra(faba5e83-7968-4013-a13f-5939d8a4bd70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70"
Jul 16 21:50:52.243931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:52.243895    2572 scope.go:117] "RemoveContainer" containerID="d35392993c12af4847df512caefca64decaf1bca83b5cc931ee3570ba0835491"
Jul 16 21:50:52.936623 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:52.936593    2572 generic.go:356] "Generic (PLEG): container finished" podID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerID="6f05cbcdeb31a15de2645930e35868adcc980f820153e03c2c6546044674565e" exitCode=7
Jul 16 21:50:52.936800 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:52.936681    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh" event={"ID":"faba5e83-7968-4013-a13f-5939d8a4bd70","Type":"ContainerDied","Data":"6f05cbcdeb31a15de2645930e35868adcc980f820153e03c2c6546044674565e"}
Jul 16 21:50:52.936800 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:52.936744    2572 scope.go:117] "RemoveContainer" containerID="d35392993c12af4847df512caefca64decaf1bca83b5cc931ee3570ba0835491"
Jul 16 21:50:52.937067 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:52.937043    2572 scope.go:117] "RemoveContainer" containerID="6f05cbcdeb31a15de2645930e35868adcc980f820153e03c2c6546044674565e"
Jul 16 21:50:52.937368 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:50:52.937332    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh_odh-ai-gateway-infra(faba5e83-7968-4013-a13f-5939d8a4bd70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70"
Jul 16 21:50:53.978092 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:53.978055    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"]
Jul 16 21:50:54.121453 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.121422    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"
Jul 16 21:50:54.208362 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.208321    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faba5e83-7968-4013-a13f-5939d8a4bd70-maas-api-service-ca\") pod \"faba5e83-7968-4013-a13f-5939d8a4bd70\" (UID: \"faba5e83-7968-4013-a13f-5939d8a4bd70\") "
Jul 16 21:50:54.208694 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.208670    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faba5e83-7968-4013-a13f-5939d8a4bd70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "faba5e83-7968-4013-a13f-5939d8a4bd70" (UID: "faba5e83-7968-4013-a13f-5939d8a4bd70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:50:54.310078 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.310028    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/faba5e83-7968-4013-a13f-5939d8a4bd70-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:50:54.509111 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509069    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"]
Jul 16 21:50:54.509585 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509568    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:50:54.509585 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509586    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:50:54.509684 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509606    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:50:54.509684 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509616    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:50:54.509684 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509627    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:54.509684 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509633    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fb84fd1-4dd3-41be-9619-8ad113dc8d24" containerName="revoke-keys"
Jul 16 21:50:54.509807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509699    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:50:54.509807 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.509714    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:50:54.512796 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.512779    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"
Jul 16 21:50:54.522152 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.522115    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"]
Jul 16 21:50:54.613109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.613014    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0dc2f0-edd7-4925-9760-6b472dcf481b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2\" (UID: \"2a0dc2f0-edd7-4925-9760-6b472dcf481b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"
Jul 16 21:50:54.714094 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.714035    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0dc2f0-edd7-4925-9760-6b472dcf481b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2\" (UID: \"2a0dc2f0-edd7-4925-9760-6b472dcf481b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"
Jul 16 21:50:54.714724 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.714700    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0dc2f0-edd7-4925-9760-6b472dcf481b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2\" (UID: \"2a0dc2f0-edd7-4925-9760-6b472dcf481b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"
Jul 16 21:50:54.824500 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.824463    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"
Jul 16 21:50:54.951555 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.951523    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"
Jul 16 21:50:54.951721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.951556    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh" event={"ID":"faba5e83-7968-4013-a13f-5939d8a4bd70","Type":"ContainerDied","Data":"e30ed8be08f7166e49561602da5f71b28e37a05d5324df0145505b638dc44430"}
Jul 16 21:50:54.951721 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.951617    2572 scope.go:117] "RemoveContainer" containerID="6f05cbcdeb31a15de2645930e35868adcc980f820153e03c2c6546044674565e"
Jul 16 21:50:54.967911 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:50:54.967885    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0dc2f0_edd7_4925_9760_6b472dcf481b.slice/crio-ed462dc84534f6594b46abbae4818207b75604929823d3876ec5a9c7124b144f WatchSource:0}: Error finding container ed462dc84534f6594b46abbae4818207b75604929823d3876ec5a9c7124b144f: Status 404 returned error can't find the container with id ed462dc84534f6594b46abbae4818207b75604929823d3876ec5a9c7124b144f
Jul 16 21:50:54.968180 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.968162    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"]
Jul 16 21:50:54.979584 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.979558    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"]
Jul 16 21:50:54.988398 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:54.987445    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-tt6lh"]
Jul 16 21:50:55.249976 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:55.249941    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" path="/var/lib/kubelet/pods/faba5e83-7968-4013-a13f-5939d8a4bd70/volumes"
Jul 16 21:50:55.959516 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:55.959423    2572 generic.go:356] "Generic (PLEG): container finished" podID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerID="c530fa51f562dad9c228addef17fbd43902043680a58baab7fd7efa94e259dac" exitCode=6
Jul 16 21:50:55.959516 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:55.959502    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2" event={"ID":"2a0dc2f0-edd7-4925-9760-6b472dcf481b","Type":"ContainerDied","Data":"c530fa51f562dad9c228addef17fbd43902043680a58baab7fd7efa94e259dac"}
Jul 16 21:50:55.959705 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:55.959546    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2" event={"ID":"2a0dc2f0-edd7-4925-9760-6b472dcf481b","Type":"ContainerStarted","Data":"ed462dc84534f6594b46abbae4818207b75604929823d3876ec5a9c7124b144f"}
Jul 16 21:50:55.959705 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:55.959692    2572 scope.go:117] "RemoveContainer" containerID="c530fa51f562dad9c228addef17fbd43902043680a58baab7fd7efa94e259dac"
Jul 16 21:50:56.967235 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:56.967143    2572 generic.go:356] "Generic (PLEG): container finished" podID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerID="e9d69055f64c9a011f36c576773dcf15ef58d818a5271f8fa9964e4042a03901" exitCode=6
Jul 16 21:50:56.967235 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:56.967224    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2" event={"ID":"2a0dc2f0-edd7-4925-9760-6b472dcf481b","Type":"ContainerDied","Data":"e9d69055f64c9a011f36c576773dcf15ef58d818a5271f8fa9964e4042a03901"}
Jul 16 21:50:56.967724 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:56.967281    2572 scope.go:117] "RemoveContainer" containerID="c530fa51f562dad9c228addef17fbd43902043680a58baab7fd7efa94e259dac"
Jul 16 21:50:56.967724 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:56.967602    2572 scope.go:117] "RemoveContainer" containerID="e9d69055f64c9a011f36c576773dcf15ef58d818a5271f8fa9964e4042a03901"
Jul 16 21:50:56.967837 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:50:56.967815    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2_odh-ai-gateway-infra(2a0dc2f0-edd7-4925-9760-6b472dcf481b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b"
Jul 16 21:50:57.975104 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:50:57.975077    2572 scope.go:117] "RemoveContainer" containerID="e9d69055f64c9a011f36c576773dcf15ef58d818a5271f8fa9964e4042a03901"
Jul 16 21:50:57.975524 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:50:57.975287    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2_odh-ai-gateway-infra(2a0dc2f0-edd7-4925-9760-6b472dcf481b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b"
Jul 16 21:51:13.244499 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:13.244460    2572 scope.go:117] "RemoveContainer" containerID="e9d69055f64c9a011f36c576773dcf15ef58d818a5271f8fa9964e4042a03901"
Jul 16 21:51:14.067125 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:14.067091    2572 generic.go:356] "Generic (PLEG): container finished" podID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerID="70ef1b9cdeaeb8c78a99bc9de1d0a4628b5853414df1a73fd954687928aec737" exitCode=6
Jul 16 21:51:14.067295 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:14.067171    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2" event={"ID":"2a0dc2f0-edd7-4925-9760-6b472dcf481b","Type":"ContainerDied","Data":"70ef1b9cdeaeb8c78a99bc9de1d0a4628b5853414df1a73fd954687928aec737"}
Jul 16 21:51:14.067295 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:14.067276    2572 scope.go:117] "RemoveContainer" containerID="e9d69055f64c9a011f36c576773dcf15ef58d818a5271f8fa9964e4042a03901"
Jul 16 21:51:14.067627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:14.067613    2572 scope.go:117] "RemoveContainer" containerID="70ef1b9cdeaeb8c78a99bc9de1d0a4628b5853414df1a73fd954687928aec737"
Jul 16 21:51:14.067875 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:51:14.067857    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2_odh-ai-gateway-infra(2a0dc2f0-edd7-4925-9760-6b472dcf481b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b"
Jul 16 21:51:15.101185 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:15.101147    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"]
Jul 16 21:51:15.239888 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:15.239861    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"
Jul 16 21:51:15.419289 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:15.419194    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0dc2f0-edd7-4925-9760-6b472dcf481b-maas-api-service-ca\") pod \"2a0dc2f0-edd7-4925-9760-6b472dcf481b\" (UID: \"2a0dc2f0-edd7-4925-9760-6b472dcf481b\") "
Jul 16 21:51:15.419653 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:15.419624    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a0dc2f0-edd7-4925-9760-6b472dcf481b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2a0dc2f0-edd7-4925-9760-6b472dcf481b" (UID: "2a0dc2f0-edd7-4925-9760-6b472dcf481b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:51:15.520338 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:15.520302    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a0dc2f0-edd7-4925-9760-6b472dcf481b-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:51:16.081313 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:16.081281    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2" event={"ID":"2a0dc2f0-edd7-4925-9760-6b472dcf481b","Type":"ContainerDied","Data":"ed462dc84534f6594b46abbae4818207b75604929823d3876ec5a9c7124b144f"}
Jul 16 21:51:16.081520 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:16.081338    2572 scope.go:117] "RemoveContainer" containerID="70ef1b9cdeaeb8c78a99bc9de1d0a4628b5853414df1a73fd954687928aec737"
Jul 16 21:51:16.081520 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:16.081294    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"
Jul 16 21:51:16.108259 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:16.108226    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"]
Jul 16 21:51:16.121778 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:16.121751    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-nggc2"]
Jul 16 21:51:17.250534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:17.250505    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" path="/var/lib/kubelet/pods/2a0dc2f0-edd7-4925-9760-6b472dcf481b/volumes"
Jul 16 21:51:33.457606 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.457509    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"]
Jul 16 21:51:33.458198 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458175    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:51:33.458198 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458197    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:51:33.458323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458222    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:51:33.458323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458231    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:51:33.458323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458244    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:51:33.458323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458253    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:51:33.458323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458261    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:51:33.458323 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458270    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:51:33.458665 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458427    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:51:33.458665 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458441    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:51:33.458665 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.458453    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="faba5e83-7968-4013-a13f-5939d8a4bd70" containerName="revoke-keys"
Jul 16 21:51:33.463140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.463096    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"
Jul 16 21:51:33.464875 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.464848    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"]
Jul 16 21:51:33.465664 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.465641    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:51:33.598681 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.598645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz\" (UID: \"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"
Jul 16 21:51:33.699172 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.699127    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz\" (UID: \"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"
Jul 16 21:51:33.699817 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.699797    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz\" (UID: \"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"
Jul 16 21:51:33.775966 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.775934    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"
Jul 16 21:51:33.918959 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:33.918935    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"]
Jul 16 21:51:33.920746 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:51:33.920713    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbbdb79d_9c88_4b4f_94f2_e38335e1e2dc.slice/crio-f33103a5fca2e7c265ad87b8ff1a8e334db9a0d4c17f26262dfb6c33e1022f21 WatchSource:0}: Error finding container f33103a5fca2e7c265ad87b8ff1a8e334db9a0d4c17f26262dfb6c33e1022f21: Status 404 returned error can't find the container with id f33103a5fca2e7c265ad87b8ff1a8e334db9a0d4c17f26262dfb6c33e1022f21
Jul 16 21:51:34.188399 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:34.188341    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz" event={"ID":"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc","Type":"ContainerStarted","Data":"f33103a5fca2e7c265ad87b8ff1a8e334db9a0d4c17f26262dfb6c33e1022f21"}
Jul 16 21:51:35.195724 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:35.195688    2572 generic.go:356] "Generic (PLEG): container finished" podID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerID="ac310628f34cb073a2ede569302a68841dfbfe2f2e5abc88a5ee0bf46766d1e8" exitCode=7
Jul 16 21:51:35.196109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:35.195777    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz" event={"ID":"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc","Type":"ContainerDied","Data":"ac310628f34cb073a2ede569302a68841dfbfe2f2e5abc88a5ee0bf46766d1e8"}
Jul 16 21:51:35.196109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:35.195934    2572 scope.go:117] "RemoveContainer" containerID="ac310628f34cb073a2ede569302a68841dfbfe2f2e5abc88a5ee0bf46766d1e8"
Jul 16 21:51:36.204100 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:36.204002    2572 generic.go:356] "Generic (PLEG): container finished" podID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerID="dbb1242b4006383b531f1aa8709c3a3d73c5ac6a7ed2d3360db7874dcddc760b" exitCode=7
Jul 16 21:51:36.204100 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:36.204045    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz" event={"ID":"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc","Type":"ContainerDied","Data":"dbb1242b4006383b531f1aa8709c3a3d73c5ac6a7ed2d3360db7874dcddc760b"}
Jul 16 21:51:36.204607 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:36.204105    2572 scope.go:117] "RemoveContainer" containerID="ac310628f34cb073a2ede569302a68841dfbfe2f2e5abc88a5ee0bf46766d1e8"
Jul 16 21:51:36.204607 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:36.204252    2572 scope.go:117] "RemoveContainer" containerID="dbb1242b4006383b531f1aa8709c3a3d73c5ac6a7ed2d3360db7874dcddc760b"
Jul 16 21:51:36.204607 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:51:36.204544    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz_odh-ai-gateway-infra(dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc"
Jul 16 21:51:37.211590 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:37.211561    2572 scope.go:117] "RemoveContainer" containerID="dbb1242b4006383b531f1aa8709c3a3d73c5ac6a7ed2d3360db7874dcddc760b"
Jul 16 21:51:37.211986 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:51:37.211788    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz_odh-ai-gateway-infra(dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc"
Jul 16 21:51:51.243698 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:51.243661    2572 scope.go:117] "RemoveContainer" containerID="dbb1242b4006383b531f1aa8709c3a3d73c5ac6a7ed2d3360db7874dcddc760b"
Jul 16 21:51:52.296470 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:52.296436    2572 generic.go:356] "Generic (PLEG): container finished" podID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerID="d1bf07e9c598fd6900c58cce91d32c6bbf993848268cba812527795742728bc9" exitCode=7
Jul 16 21:51:52.296925 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:52.296514    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz" event={"ID":"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc","Type":"ContainerDied","Data":"d1bf07e9c598fd6900c58cce91d32c6bbf993848268cba812527795742728bc9"}
Jul 16 21:51:52.296925 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:52.296575    2572 scope.go:117] "RemoveContainer" containerID="dbb1242b4006383b531f1aa8709c3a3d73c5ac6a7ed2d3360db7874dcddc760b"
Jul 16 21:51:52.296925 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:52.296870    2572 scope.go:117] "RemoveContainer" containerID="d1bf07e9c598fd6900c58cce91d32c6bbf993848268cba812527795742728bc9"
Jul 16 21:51:52.297139 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:51:52.297116    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz_odh-ai-gateway-infra(dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc"
Jul 16 21:51:53.332494 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:53.332458    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"]
Jul 16 21:51:53.475978 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:53.475944    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"
Jul 16 21:51:53.595340 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:53.595239    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc-maas-api-service-ca\") pod \"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc\" (UID: \"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc\") "
Jul 16 21:51:53.595641 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:53.595619    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" (UID: "dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:51:53.696828 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:53.696776    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:51:54.311179 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:54.311152    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"
Jul 16 21:51:54.311179 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:54.311168    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz" event={"ID":"dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc","Type":"ContainerDied","Data":"f33103a5fca2e7c265ad87b8ff1a8e334db9a0d4c17f26262dfb6c33e1022f21"}
Jul 16 21:51:54.311453 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:54.311227    2572 scope.go:117] "RemoveContainer" containerID="d1bf07e9c598fd6900c58cce91d32c6bbf993848268cba812527795742728bc9"
Jul 16 21:51:54.343524 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:54.343489    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"]
Jul 16 21:51:54.355050 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:54.355019    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-g4kcz"]
Jul 16 21:51:55.249637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:51:55.249606    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" path="/var/lib/kubelet/pods/dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc/volumes"
Jul 16 21:52:05.021389 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.021333    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"]
Jul 16 21:52:05.021994 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.021937    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:05.021994 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.021961    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:05.022118 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.022004    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:05.022118 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.022012    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:05.022118 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.022115    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:05.022270 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.022129    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:05.022270 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.022140    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:05.022270 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.022152    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a0dc2f0-edd7-4925-9760-6b472dcf481b" containerName="revoke-keys"
Jul 16 21:52:05.026504 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.026480    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"
Jul 16 21:52:05.029283 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.029264    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:52:05.034332 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.034316    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"]
Jul 16 21:52:05.100229 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.100191    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8\" (UID: \"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"
Jul 16 21:52:05.201462 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.201418    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8\" (UID: \"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"
Jul 16 21:52:05.202067 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.202048    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8\" (UID: \"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"
Jul 16 21:52:05.338673 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.338615    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"
Jul 16 21:52:05.483981 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:05.483954    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"]
Jul 16 21:52:05.486646 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:52:05.486621    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b84f9f9_a5d1_49f1_8f9b_d33fdc43ef02.slice/crio-4e0b60463d4c4b86258bedd5931106260848d11b6dd7d07919f57f6ef6b541c1 WatchSource:0}: Error finding container 4e0b60463d4c4b86258bedd5931106260848d11b6dd7d07919f57f6ef6b541c1: Status 404 returned error can't find the container with id 4e0b60463d4c4b86258bedd5931106260848d11b6dd7d07919f57f6ef6b541c1
Jul 16 21:52:06.381504 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:06.381414    2572 generic.go:356] "Generic (PLEG): container finished" podID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerID="2cc66b9c227fa4b805566f60cbdaaada865bd04bf71c179f5bdfc837ee41d4ec" exitCode=6
Jul 16 21:52:06.381931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:06.381504    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8" event={"ID":"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02","Type":"ContainerDied","Data":"2cc66b9c227fa4b805566f60cbdaaada865bd04bf71c179f5bdfc837ee41d4ec"}
Jul 16 21:52:06.381931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:06.381556    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8" event={"ID":"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02","Type":"ContainerStarted","Data":"4e0b60463d4c4b86258bedd5931106260848d11b6dd7d07919f57f6ef6b541c1"}
Jul 16 21:52:06.381931 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:06.381652    2572 scope.go:117] "RemoveContainer" containerID="2cc66b9c227fa4b805566f60cbdaaada865bd04bf71c179f5bdfc837ee41d4ec"
Jul 16 21:52:07.388949 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:07.388916    2572 generic.go:356] "Generic (PLEG): container finished" podID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerID="01cd1ddff554d3293aed0e39d53b38848832fd6b579471539a288258ca2f6cfe" exitCode=6
Jul 16 21:52:07.389346 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:07.388986    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8" event={"ID":"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02","Type":"ContainerDied","Data":"01cd1ddff554d3293aed0e39d53b38848832fd6b579471539a288258ca2f6cfe"}
Jul 16 21:52:07.389346 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:07.389030    2572 scope.go:117] "RemoveContainer" containerID="2cc66b9c227fa4b805566f60cbdaaada865bd04bf71c179f5bdfc837ee41d4ec"
Jul 16 21:52:07.389346 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:07.389219    2572 scope.go:117] "RemoveContainer" containerID="01cd1ddff554d3293aed0e39d53b38848832fd6b579471539a288258ca2f6cfe"
Jul 16 21:52:07.389524 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:52:07.389453    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8_odh-ai-gateway-infra(6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02"
Jul 16 21:52:08.396999 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:08.396971    2572 scope.go:117] "RemoveContainer" containerID="01cd1ddff554d3293aed0e39d53b38848832fd6b579471539a288258ca2f6cfe"
Jul 16 21:52:08.397439 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:52:08.397170    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8_odh-ai-gateway-infra(6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02"
Jul 16 21:52:22.244220 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:22.244180    2572 scope.go:117] "RemoveContainer" containerID="01cd1ddff554d3293aed0e39d53b38848832fd6b579471539a288258ca2f6cfe"
Jul 16 21:52:23.491249 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:23.491215    2572 generic.go:356] "Generic (PLEG): container finished" podID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerID="03d383d45887fcdb14087d9445157661c47720cf3c45a8793b8d25d42757d98b" exitCode=6
Jul 16 21:52:23.491745 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:23.491293    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8" event={"ID":"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02","Type":"ContainerDied","Data":"03d383d45887fcdb14087d9445157661c47720cf3c45a8793b8d25d42757d98b"}
Jul 16 21:52:23.491745 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:23.491383    2572 scope.go:117] "RemoveContainer" containerID="01cd1ddff554d3293aed0e39d53b38848832fd6b579471539a288258ca2f6cfe"
Jul 16 21:52:23.491745 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:23.491676    2572 scope.go:117] "RemoveContainer" containerID="03d383d45887fcdb14087d9445157661c47720cf3c45a8793b8d25d42757d98b"
Jul 16 21:52:23.491950 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:52:23.491930    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8_odh-ai-gateway-infra(6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02"
Jul 16 21:52:24.520716 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:24.520680    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"]
Jul 16 21:52:24.661468 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:24.661442    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"
Jul 16 21:52:24.785344 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:24.785305    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02-maas-api-service-ca\") pod \"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02\" (UID: \"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02\") "
Jul 16 21:52:24.785666 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:24.785645    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" (UID: "6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:52:24.887062 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:24.887026    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:52:25.505982 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:25.505953    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"
Jul 16 21:52:25.506169 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:25.505978    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8" event={"ID":"6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02","Type":"ContainerDied","Data":"4e0b60463d4c4b86258bedd5931106260848d11b6dd7d07919f57f6ef6b541c1"}
Jul 16 21:52:25.506169 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:25.506047    2572 scope.go:117] "RemoveContainer" containerID="03d383d45887fcdb14087d9445157661c47720cf3c45a8793b8d25d42757d98b"
Jul 16 21:52:25.527220 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:25.525197    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"]
Jul 16 21:52:25.534204 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:25.534165    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-jrtz8"]
Jul 16 21:52:27.249940 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:27.249905    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" path="/var/lib/kubelet/pods/6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02/volumes"
Jul 16 21:52:33.506967 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.506927    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"]
Jul 16 21:52:33.507397 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.507381    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:33.507397 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.507399    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbbdb79d-9c88-4b4f-94f2-e38335e1e2dc" containerName="revoke-keys"
Jul 16 21:52:33.507480 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.507409    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:52:33.507480 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.507415    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:52:33.507480 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.507429    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:52:33.507480 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.507434    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:52:33.507610 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.507505    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:52:33.507610 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.507513    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:52:33.512006 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.511972    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"
Jul 16 21:52:33.515774 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.515712    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:52:33.518653 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.518626    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"]
Jul 16 21:52:33.671820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.671784    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d8d178f-8d2d-4eea-9dbb-d54349facf5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc\" (UID: \"4d8d178f-8d2d-4eea-9dbb-d54349facf5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"
Jul 16 21:52:33.773061 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.773029    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d8d178f-8d2d-4eea-9dbb-d54349facf5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc\" (UID: \"4d8d178f-8d2d-4eea-9dbb-d54349facf5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"
Jul 16 21:52:33.773713 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.773691    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d8d178f-8d2d-4eea-9dbb-d54349facf5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc\" (UID: \"4d8d178f-8d2d-4eea-9dbb-d54349facf5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"
Jul 16 21:52:33.827079 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.827039    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"
Jul 16 21:52:33.970572 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:33.970542    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"]
Jul 16 21:52:33.972175 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:52:33.972149    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8d178f_8d2d_4eea_9dbb_d54349facf5b.slice/crio-53bfcb60b1342c30e6497ab92d761b42336652f7c13424c6d960ab55cb0fbc95 WatchSource:0}: Error finding container 53bfcb60b1342c30e6497ab92d761b42336652f7c13424c6d960ab55cb0fbc95: Status 404 returned error can't find the container with id 53bfcb60b1342c30e6497ab92d761b42336652f7c13424c6d960ab55cb0fbc95
Jul 16 21:52:34.557937 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:34.557902    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" event={"ID":"4d8d178f-8d2d-4eea-9dbb-d54349facf5b","Type":"ContainerStarted","Data":"1ff627993f1d28f55ba5092c9a02240ac044835f9b5911af30b9649dee8ac55c"}
Jul 16 21:52:34.558370 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:34.557956    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" event={"ID":"4d8d178f-8d2d-4eea-9dbb-d54349facf5b","Type":"ContainerStarted","Data":"53bfcb60b1342c30e6497ab92d761b42336652f7c13424c6d960ab55cb0fbc95"}
Jul 16 21:52:34.579172 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:34.579115    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" podStartSLOduration=1.579099013 podStartE2EDuration="1.579099013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:52:34.577380579 +0000 UTC m=+2220.088540311" watchObservedRunningTime="2026-07-16 21:52:34.579099013 +0000 UTC m=+2220.090258780"
Jul 16 21:52:35.565302 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:35.565271    2572 generic.go:356] "Generic (PLEG): container finished" podID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerID="1ff627993f1d28f55ba5092c9a02240ac044835f9b5911af30b9649dee8ac55c" exitCode=7
Jul 16 21:52:35.565710 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:35.565372    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" event={"ID":"4d8d178f-8d2d-4eea-9dbb-d54349facf5b","Type":"ContainerDied","Data":"1ff627993f1d28f55ba5092c9a02240ac044835f9b5911af30b9649dee8ac55c"}
Jul 16 21:52:35.565710 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:35.565583    2572 scope.go:117] "RemoveContainer" containerID="1ff627993f1d28f55ba5092c9a02240ac044835f9b5911af30b9649dee8ac55c"
Jul 16 21:52:36.572790 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:36.572756    2572 generic.go:356] "Generic (PLEG): container finished" podID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerID="3393da5bc5a02d0ee0dbe859ca7811ceb4b94b3da631622295d49a62e3cdafc1" exitCode=7
Jul 16 21:52:36.573295 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:36.572844    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" event={"ID":"4d8d178f-8d2d-4eea-9dbb-d54349facf5b","Type":"ContainerDied","Data":"3393da5bc5a02d0ee0dbe859ca7811ceb4b94b3da631622295d49a62e3cdafc1"}
Jul 16 21:52:36.573295 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:36.572897    2572 scope.go:117] "RemoveContainer" containerID="1ff627993f1d28f55ba5092c9a02240ac044835f9b5911af30b9649dee8ac55c"
Jul 16 21:52:36.573295 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:36.573023    2572 scope.go:117] "RemoveContainer" containerID="3393da5bc5a02d0ee0dbe859ca7811ceb4b94b3da631622295d49a62e3cdafc1"
Jul 16 21:52:36.573295 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:52:36.573267    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc_odh-ai-gateway-infra(4d8d178f-8d2d-4eea-9dbb-d54349facf5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b"
Jul 16 21:52:36.643970 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:36.643940    2572 scope.go:117] "RemoveContainer" containerID="8cd6dbce7c5bd9396b5f64916180b63db5d6ea1f871408e3f70b6c6cb02c8bc1"
Jul 16 21:52:37.580099 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:37.580073    2572 scope.go:117] "RemoveContainer" containerID="3393da5bc5a02d0ee0dbe859ca7811ceb4b94b3da631622295d49a62e3cdafc1"
Jul 16 21:52:37.580577 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:52:37.580322    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc_odh-ai-gateway-infra(4d8d178f-8d2d-4eea-9dbb-d54349facf5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b"
Jul 16 21:52:52.244420 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:52.244342    2572 scope.go:117] "RemoveContainer" containerID="3393da5bc5a02d0ee0dbe859ca7811ceb4b94b3da631622295d49a62e3cdafc1"
Jul 16 21:52:52.664936 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:52.664903    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" event={"ID":"4d8d178f-8d2d-4eea-9dbb-d54349facf5b","Type":"ContainerStarted","Data":"9b55d9c5be7a4584bc81d64386f1649c7142fac014f490d47508b3dde7b8f943"}
Jul 16 21:52:53.672619 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:53.672584    2572 generic.go:356] "Generic (PLEG): container finished" podID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerID="9b55d9c5be7a4584bc81d64386f1649c7142fac014f490d47508b3dde7b8f943" exitCode=7
Jul 16 21:52:53.672619 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:53.672626    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" event={"ID":"4d8d178f-8d2d-4eea-9dbb-d54349facf5b","Type":"ContainerDied","Data":"9b55d9c5be7a4584bc81d64386f1649c7142fac014f490d47508b3dde7b8f943"}
Jul 16 21:52:53.673193 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:53.672684    2572 scope.go:117] "RemoveContainer" containerID="3393da5bc5a02d0ee0dbe859ca7811ceb4b94b3da631622295d49a62e3cdafc1"
Jul 16 21:52:53.673193 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:53.672948    2572 scope.go:117] "RemoveContainer" containerID="9b55d9c5be7a4584bc81d64386f1649c7142fac014f490d47508b3dde7b8f943"
Jul 16 21:52:53.673193 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:52:53.673171    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc_odh-ai-gateway-infra(4d8d178f-8d2d-4eea-9dbb-d54349facf5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b"
Jul 16 21:52:53.700900 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:53.700866    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"]
Jul 16 21:52:54.820863 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:54.820834    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"
Jul 16 21:52:54.870855 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:54.870819    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d8d178f-8d2d-4eea-9dbb-d54349facf5b-maas-api-service-ca\") pod \"4d8d178f-8d2d-4eea-9dbb-d54349facf5b\" (UID: \"4d8d178f-8d2d-4eea-9dbb-d54349facf5b\") "
Jul 16 21:52:54.871174 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:54.871152    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d8d178f-8d2d-4eea-9dbb-d54349facf5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d8d178f-8d2d-4eea-9dbb-d54349facf5b" (UID: "4d8d178f-8d2d-4eea-9dbb-d54349facf5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:52:54.972495 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:54.972399    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d8d178f-8d2d-4eea-9dbb-d54349facf5b-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:52:55.686529 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:55.686494    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc" event={"ID":"4d8d178f-8d2d-4eea-9dbb-d54349facf5b","Type":"ContainerDied","Data":"53bfcb60b1342c30e6497ab92d761b42336652f7c13424c6d960ab55cb0fbc95"}
Jul 16 21:52:55.686728 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:55.686557    2572 scope.go:117] "RemoveContainer" containerID="9b55d9c5be7a4584bc81d64386f1649c7142fac014f490d47508b3dde7b8f943"
Jul 16 21:52:55.686728 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:55.686503    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"
Jul 16 21:52:55.705218 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:55.705187    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"]
Jul 16 21:52:55.716770 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:55.716740    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-msjkc"]
Jul 16 21:52:57.249994 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:52:57.249914    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" path="/var/lib/kubelet/pods/4d8d178f-8d2d-4eea-9dbb-d54349facf5b/volumes"
Jul 16 21:53:05.070403 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070362    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"]
Jul 16 21:53:05.070795 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070755    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:05.070795 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070765    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:05.070795 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070776    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:53:05.070795 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070782    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:53:05.070932 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070801    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:05.070932 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070807    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:05.070932 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070867    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:05.070932 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070874    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b84f9f9-a5d1-49f1-8f9b-d33fdc43ef02" containerName="revoke-keys"
Jul 16 21:53:05.070932 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.070884    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:05.075203 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.075180    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"
Jul 16 21:53:05.077966 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.077946    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:53:05.083423 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.082937    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"]
Jul 16 21:53:05.171739 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.171698    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/123a1044-84b4-4721-86d1-b96696075576-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln\" (UID: \"123a1044-84b4-4721-86d1-b96696075576\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"
Jul 16 21:53:05.272849 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.272811    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/123a1044-84b4-4721-86d1-b96696075576-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln\" (UID: \"123a1044-84b4-4721-86d1-b96696075576\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"
Jul 16 21:53:05.273461 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.273441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/123a1044-84b4-4721-86d1-b96696075576-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln\" (UID: \"123a1044-84b4-4721-86d1-b96696075576\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"
Jul 16 21:53:05.387257 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.387159    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"
Jul 16 21:53:05.542239 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.542214    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"]
Jul 16 21:53:05.543793 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:53:05.543771    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123a1044_84b4_4721_86d1_b96696075576.slice/crio-0b4c167197892101a6603eb82429638dde40c3fb0a2919798829c0a113cd0718 WatchSource:0}: Error finding container 0b4c167197892101a6603eb82429638dde40c3fb0a2919798829c0a113cd0718: Status 404 returned error can't find the container with id 0b4c167197892101a6603eb82429638dde40c3fb0a2919798829c0a113cd0718
Jul 16 21:53:05.746584 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:05.746549    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" event={"ID":"123a1044-84b4-4721-86d1-b96696075576","Type":"ContainerStarted","Data":"0b4c167197892101a6603eb82429638dde40c3fb0a2919798829c0a113cd0718"}
Jul 16 21:53:06.753892 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:06.753855    2572 generic.go:356] "Generic (PLEG): container finished" podID="123a1044-84b4-4721-86d1-b96696075576" containerID="3b0b2e05ad1f0e33e8e53474127f0e27eb6e5826c5982b6eb02d8cb4115063d3" exitCode=6
Jul 16 21:53:06.754279 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:06.753917    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" event={"ID":"123a1044-84b4-4721-86d1-b96696075576","Type":"ContainerDied","Data":"3b0b2e05ad1f0e33e8e53474127f0e27eb6e5826c5982b6eb02d8cb4115063d3"}
Jul 16 21:53:06.754279 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:06.754161    2572 scope.go:117] "RemoveContainer" containerID="3b0b2e05ad1f0e33e8e53474127f0e27eb6e5826c5982b6eb02d8cb4115063d3"
Jul 16 21:53:07.761057 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:07.760971    2572 generic.go:356] "Generic (PLEG): container finished" podID="123a1044-84b4-4721-86d1-b96696075576" containerID="17c5338b857acfb772a6a699189ffdcd8bee3e1f7cd5dcd5898c389b8c539427" exitCode=6
Jul 16 21:53:07.761476 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:07.761043    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" event={"ID":"123a1044-84b4-4721-86d1-b96696075576","Type":"ContainerDied","Data":"17c5338b857acfb772a6a699189ffdcd8bee3e1f7cd5dcd5898c389b8c539427"}
Jul 16 21:53:07.761476 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:07.761115    2572 scope.go:117] "RemoveContainer" containerID="3b0b2e05ad1f0e33e8e53474127f0e27eb6e5826c5982b6eb02d8cb4115063d3"
Jul 16 21:53:07.761476 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:07.761332    2572 scope.go:117] "RemoveContainer" containerID="17c5338b857acfb772a6a699189ffdcd8bee3e1f7cd5dcd5898c389b8c539427"
Jul 16 21:53:07.761626 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:07.761596    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln_odh-ai-gateway-infra(123a1044-84b4-4721-86d1-b96696075576)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" podUID="123a1044-84b4-4721-86d1-b96696075576"
Jul 16 21:53:08.767930 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:08.767901    2572 scope.go:117] "RemoveContainer" containerID="17c5338b857acfb772a6a699189ffdcd8bee3e1f7cd5dcd5898c389b8c539427"
Jul 16 21:53:08.768373 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:08.768126    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln_odh-ai-gateway-infra(123a1044-84b4-4721-86d1-b96696075576)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" podUID="123a1044-84b4-4721-86d1-b96696075576"
Jul 16 21:53:23.244295 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:23.244254    2572 scope.go:117] "RemoveContainer" containerID="17c5338b857acfb772a6a699189ffdcd8bee3e1f7cd5dcd5898c389b8c539427"
Jul 16 21:53:23.857140 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:23.857104    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" event={"ID":"123a1044-84b4-4721-86d1-b96696075576","Type":"ContainerStarted","Data":"d57413fcfe80960976af372019f52bcdfa3f9c59bff1401975e899b9c53a991a"}
Jul 16 21:53:23.875210 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:23.875157    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" podStartSLOduration=18.875140264 podStartE2EDuration="18.875140264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:53:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:53:23.87416346 +0000 UTC m=+2269.385323195" watchObservedRunningTime="2026-07-16 21:53:23.875140264 +0000 UTC m=+2269.386300047"
Jul 16 21:53:24.863976 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:24.863944    2572 generic.go:356] "Generic (PLEG): container finished" podID="123a1044-84b4-4721-86d1-b96696075576" containerID="d57413fcfe80960976af372019f52bcdfa3f9c59bff1401975e899b9c53a991a" exitCode=6
Jul 16 21:53:24.864532 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:24.863997    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" event={"ID":"123a1044-84b4-4721-86d1-b96696075576","Type":"ContainerDied","Data":"d57413fcfe80960976af372019f52bcdfa3f9c59bff1401975e899b9c53a991a"}
Jul 16 21:53:24.864532 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:24.864044    2572 scope.go:117] "RemoveContainer" containerID="17c5338b857acfb772a6a699189ffdcd8bee3e1f7cd5dcd5898c389b8c539427"
Jul 16 21:53:24.864532 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:24.864284    2572 scope.go:117] "RemoveContainer" containerID="d57413fcfe80960976af372019f52bcdfa3f9c59bff1401975e899b9c53a991a"
Jul 16 21:53:24.864702 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:24.864565    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln_odh-ai-gateway-infra(123a1044-84b4-4721-86d1-b96696075576)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" podUID="123a1044-84b4-4721-86d1-b96696075576"
Jul 16 21:53:24.891898 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:24.891861    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"]
Jul 16 21:53:26.000804 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.000770    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"
Jul 16 21:53:26.075474 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.075441    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/123a1044-84b4-4721-86d1-b96696075576-maas-api-service-ca\") pod \"123a1044-84b4-4721-86d1-b96696075576\" (UID: \"123a1044-84b4-4721-86d1-b96696075576\") "
Jul 16 21:53:26.075873 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.075848    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/123a1044-84b4-4721-86d1-b96696075576-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "123a1044-84b4-4721-86d1-b96696075576" (UID: "123a1044-84b4-4721-86d1-b96696075576"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:26.176414 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.176301    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/123a1044-84b4-4721-86d1-b96696075576-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:26.877013 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.876978    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln" event={"ID":"123a1044-84b4-4721-86d1-b96696075576","Type":"ContainerDied","Data":"0b4c167197892101a6603eb82429638dde40c3fb0a2919798829c0a113cd0718"}
Jul 16 21:53:26.877013 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.876999    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"
Jul 16 21:53:26.877272 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.877046    2572 scope.go:117] "RemoveContainer" containerID="d57413fcfe80960976af372019f52bcdfa3f9c59bff1401975e899b9c53a991a"
Jul 16 21:53:26.904586 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.904553    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"]
Jul 16 21:53:26.910645 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:26.910611    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fa7d409d-rt2ln"]
Jul 16 21:53:27.249946 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:27.249863    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="123a1044-84b4-4721-86d1-b96696075576" path="/var/lib/kubelet/pods/123a1044-84b4-4721-86d1-b96696075576/volumes"
Jul 16 21:53:33.440161 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.440108    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"]
Jul 16 21:53:33.440706 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.440387    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy" containerID="cri-o://7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8" gracePeriod=30
Jul 16 21:53:33.563309 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563265    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"]
Jul 16 21:53:33.563923 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563897    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:33.563923 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563922    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:33.564105 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563939    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:33.564105 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563949    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:33.564105 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563959    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:33.564105 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563968    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:33.564105 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563981    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:33.564105 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.563989    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:33.564105 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.564098    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:33.564515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.564110    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:33.564515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.564125    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d8d178f-8d2d-4eea-9dbb-d54349facf5b" containerName="revoke-keys"
Jul 16 21:53:33.569122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.569094    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"
Jul 16 21:53:33.572909 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.572886    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:53:33.575612 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.575583    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"]
Jul 16 21:53:33.643993 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.643958    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6285aa9-5f19-49fc-8142-63f0b1faee34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7\" (UID: \"c6285aa9-5f19-49fc-8142-63f0b1faee34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"
Jul 16 21:53:33.744639 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.744548    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6285aa9-5f19-49fc-8142-63f0b1faee34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7\" (UID: \"c6285aa9-5f19-49fc-8142-63f0b1faee34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"
Jul 16 21:53:33.745180 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.745158    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6285aa9-5f19-49fc-8142-63f0b1faee34-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7\" (UID: \"c6285aa9-5f19-49fc-8142-63f0b1faee34\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"
Jul 16 21:53:33.889590 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:33.889556    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"
Jul 16 21:53:34.027861 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:34.027816    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"]
Jul 16 21:53:34.033803 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:53:34.033773    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6285aa9_5f19_49fc_8142_63f0b1faee34.slice/crio-e8d78b139204cc5b3b62f17cc6680e1c9ed7344044ffb31d801b464364981521 WatchSource:0}: Error finding container e8d78b139204cc5b3b62f17cc6680e1c9ed7344044ffb31d801b464364981521: Status 404 returned error can't find the container with id e8d78b139204cc5b3b62f17cc6680e1c9ed7344044ffb31d801b464364981521
Jul 16 21:53:34.923865 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:34.923778    2572 generic.go:356] "Generic (PLEG): container finished" podID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerID="7bc4387aa747580aa949f0308d1fb8786f49ba7b18f51660cdda8b366b2a134f" exitCode=7
Jul 16 21:53:34.924275 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:34.923865    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7" event={"ID":"c6285aa9-5f19-49fc-8142-63f0b1faee34","Type":"ContainerDied","Data":"7bc4387aa747580aa949f0308d1fb8786f49ba7b18f51660cdda8b366b2a134f"}
Jul 16 21:53:34.924275 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:34.923917    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7" event={"ID":"c6285aa9-5f19-49fc-8142-63f0b1faee34","Type":"ContainerStarted","Data":"e8d78b139204cc5b3b62f17cc6680e1c9ed7344044ffb31d801b464364981521"}
Jul 16 21:53:34.924275 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:34.924024    2572 scope.go:117] "RemoveContainer" containerID="7bc4387aa747580aa949f0308d1fb8786f49ba7b18f51660cdda8b366b2a134f"
Jul 16 21:53:35.932070 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:35.931974    2572 generic.go:356] "Generic (PLEG): container finished" podID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerID="b95cdec378ba1f98ccb7bd15385d0bf7629c013378d9a00aee8146fa6fd46047" exitCode=7
Jul 16 21:53:35.932514 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:35.932059    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7" event={"ID":"c6285aa9-5f19-49fc-8142-63f0b1faee34","Type":"ContainerDied","Data":"b95cdec378ba1f98ccb7bd15385d0bf7629c013378d9a00aee8146fa6fd46047"}
Jul 16 21:53:35.932514 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:35.932128    2572 scope.go:117] "RemoveContainer" containerID="7bc4387aa747580aa949f0308d1fb8786f49ba7b18f51660cdda8b366b2a134f"
Jul 16 21:53:35.932514 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:35.932234    2572 scope.go:117] "RemoveContainer" containerID="b95cdec378ba1f98ccb7bd15385d0bf7629c013378d9a00aee8146fa6fd46047"
Jul 16 21:53:35.932637 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:35.932529    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7_odh-ai-gateway-infra(c6285aa9-5f19-49fc-8142-63f0b1faee34)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34"
Jul 16 21:53:36.940204 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:36.940130    2572 scope.go:117] "RemoveContainer" containerID="b95cdec378ba1f98ccb7bd15385d0bf7629c013378d9a00aee8146fa6fd46047"
Jul 16 21:53:36.940590 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:36.940336    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7_odh-ai-gateway-infra(c6285aa9-5f19-49fc-8142-63f0b1faee34)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34"
Jul 16 21:53:38.713058 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.712973    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:53:38.793924 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.793890    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-socket\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.793924 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.793925    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-data\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.794182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.793961    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-envoy\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.794182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.793983    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-token\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.794182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794001    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-credential-socket\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.794182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794076    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-certs\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.794182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794157    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:53:38.794549 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794213    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istiod-ca-cert\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.794549 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794258    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc96h\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-kube-api-access-bc96h\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.794549 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794312    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-podinfo\") pod \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\" (UID: \"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05\") "
Jul 16 21:53:38.794549 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794314    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:53:38.794549 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794329    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:53:38.794815 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794591    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:38.794815 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794688    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-socket\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.794815 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794706    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-credential-socket\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.794815 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794723    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-workload-certs\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.794815 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.794738    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istiod-ca-cert\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.795278 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.795131    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-data" (OuterVolumeSpecName: "istio-data") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:53:38.796569 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.796542    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-kube-api-access-bc96h" (OuterVolumeSpecName: "kube-api-access-bc96h") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "kube-api-access-bc96h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:53:38.796774 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.796751    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:53:38.796915 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.796777    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:53:38.797178 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.797157    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-token" (OuterVolumeSpecName: "istio-token") pod "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" (UID: "92dcdb0f-9eac-4bfb-95e4-76bcb3327e05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:53:38.896122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.896081    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc96h\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-kube-api-access-bc96h\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.896122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.896117    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-podinfo\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.896122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.896127    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-data\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.896122 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.896134    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-envoy\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.896435 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.896142    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05-istio-token\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:38.953447 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.953414    2572 generic.go:356] "Generic (PLEG): container finished" podID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerID="7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8" exitCode=0
Jul 16 21:53:38.953618 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.953467    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"
Jul 16 21:53:38.953618 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.953505    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" event={"ID":"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05","Type":"ContainerDied","Data":"7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8"}
Jul 16 21:53:38.953618 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.953561    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b" event={"ID":"92dcdb0f-9eac-4bfb-95e4-76bcb3327e05","Type":"ContainerDied","Data":"fdac65129169a804487b66f4c41f113b1ca7e948a83640e8bc12b83cb0825492"}
Jul 16 21:53:38.953618 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.953586    2572 scope.go:117] "RemoveContainer" containerID="7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8"
Jul 16 21:53:38.962974 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.962954    2572 scope.go:117] "RemoveContainer" containerID="1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46"
Jul 16 21:53:38.972112 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.972088    2572 scope.go:117] "RemoveContainer" containerID="7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8"
Jul 16 21:53:38.972453 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:38.972430    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8\": container with ID starting with 7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8 not found: ID does not exist" containerID="7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8"
Jul 16 21:53:38.972534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.972461    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8"} err="failed to get container status \"7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8\": rpc error: code = NotFound desc = could not find container \"7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8\": container with ID starting with 7a2a3d3d1193ec042189f0ac80a996f2b0f08abc447d0f54a573035b5cd1e7b8 not found: ID does not exist"
Jul 16 21:53:38.972534 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.972476    2572 scope.go:117] "RemoveContainer" containerID="1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46"
Jul 16 21:53:38.972745 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:38.972721    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46\": container with ID starting with 1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46 not found: ID does not exist" containerID="1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46"
Jul 16 21:53:38.972802 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.972751    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46"} err="failed to get container status \"1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46\": rpc error: code = NotFound desc = could not find container \"1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46\": container with ID starting with 1cab62556ec502f9547e07c0985b7aa85bd33ad07d8181c30a50211d640d9e46 not found: ID does not exist"
Jul 16 21:53:38.979254 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.979223    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"]
Jul 16 21:53:38.983159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:38.983133    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f6d0b514-openshift-default-dc568bc95-stw4b"]
Jul 16 21:53:39.250540 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:39.250465    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" path="/var/lib/kubelet/pods/92dcdb0f-9eac-4bfb-95e4-76bcb3327e05/volumes"
Jul 16 21:53:50.243565 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:50.243528    2572 scope.go:117] "RemoveContainer" containerID="b95cdec378ba1f98ccb7bd15385d0bf7629c013378d9a00aee8146fa6fd46047"
Jul 16 21:53:51.024161 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:51.024131    2572 generic.go:356] "Generic (PLEG): container finished" podID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerID="703940e31dfca0a65dbd1d1b3ea580956a01e9c3ada2f60812d1c04dea74e8fe" exitCode=6
Jul 16 21:53:51.024330 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:51.024208    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7" event={"ID":"c6285aa9-5f19-49fc-8142-63f0b1faee34","Type":"ContainerDied","Data":"703940e31dfca0a65dbd1d1b3ea580956a01e9c3ada2f60812d1c04dea74e8fe"}
Jul 16 21:53:51.024330 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:51.024273    2572 scope.go:117] "RemoveContainer" containerID="b95cdec378ba1f98ccb7bd15385d0bf7629c013378d9a00aee8146fa6fd46047"
Jul 16 21:53:51.024590 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:51.024563    2572 scope.go:117] "RemoveContainer" containerID="703940e31dfca0a65dbd1d1b3ea580956a01e9c3ada2f60812d1c04dea74e8fe"
Jul 16 21:53:51.024810 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:51.024789    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7_odh-ai-gateway-infra(c6285aa9-5f19-49fc-8142-63f0b1faee34)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34"
Jul 16 21:53:51.812984 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:51.812956    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f4k5g_82f32722-26a1-4261-a07d-a07ead89c7c5/manager/0.log"
Jul 16 21:53:51.925652 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:51.925621    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dcswj_d7626221-53db-424f-abfb-958a90cc6f03/manager/0.log"
Jul 16 21:53:52.046541 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.046510    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5659cfddb-8rcp4_a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a/manager/0.log"
Jul 16 21:53:52.052637 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.052606    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"]
Jul 16 21:53:52.156155 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.156129    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n4s9m_bbee227a-765d-4d7d-87fd-ccad2a11870e/server/0.log"
Jul 16 21:53:52.208316 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.208289    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"
Jul 16 21:53:52.277695 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.277663    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-96f2w_c30d7193-e264-40d2-959e-7997f27c9760/manager/0.log"
Jul 16 21:53:52.327604 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.327564    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6285aa9-5f19-49fc-8142-63f0b1faee34-maas-api-service-ca\") pod \"c6285aa9-5f19-49fc-8142-63f0b1faee34\" (UID: \"c6285aa9-5f19-49fc-8142-63f0b1faee34\") "
Jul 16 21:53:52.327943 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.327921    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6285aa9-5f19-49fc-8142-63f0b1faee34-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6285aa9-5f19-49fc-8142-63f0b1faee34" (UID: "c6285aa9-5f19-49fc-8142-63f0b1faee34"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:53:52.428687 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.428601    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6285aa9-5f19-49fc-8142-63f0b1faee34-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:53:52.499881 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.499852    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4b6cd7d5-947rd_bb765821-10ed-48ae-8b06-2472a6b9aea9/copy-manifests/0.log"
Jul 16 21:53:52.506780 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:52.506757    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4b6cd7d5-947rd_bb765821-10ed-48ae-8b06-2472a6b9aea9/manager/0.log"
Jul 16 21:53:53.037566 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:53.037539    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"
Jul 16 21:53:53.037566 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:53.037559    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7" event={"ID":"c6285aa9-5f19-49fc-8142-63f0b1faee34","Type":"ContainerDied","Data":"e8d78b139204cc5b3b62f17cc6680e1c9ed7344044ffb31d801b464364981521"}
Jul 16 21:53:53.038057 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:53.037612    2572 scope.go:117] "RemoveContainer" containerID="703940e31dfca0a65dbd1d1b3ea580956a01e9c3ada2f60812d1c04dea74e8fe"
Jul 16 21:53:53.067360 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:53.067315    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"]
Jul 16 21:53:53.074311 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:53.074274    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-2czq7"]
Jul 16 21:53:53.098763 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:53.098735    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737290-s29qk_33a8a971-8011-4119-8e07-53c4a492ea44/cleanup/0.log"
Jul 16 21:53:53.250342 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:53.250309    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" path="/var/lib/kubelet/pods/c6285aa9-5f19-49fc-8142-63f0b1faee34/volumes"
Jul 16 21:53:53.420382 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:53.420253    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-nwdcj_28c36354-901b-4cb9-ad5a-5ce27c091652/postgres/0.log"
Jul 16 21:53:54.169412 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.169385    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8_b6d8de27-f4d2-48c6-8e21-28883d5025cd/util/0.log"
Jul 16 21:53:54.176037 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.176018    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8_b6d8de27-f4d2-48c6-8e21-28883d5025cd/pull/0.log"
Jul 16 21:53:54.187417 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.187392    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8_b6d8de27-f4d2-48c6-8e21-28883d5025cd/extract/0.log"
Jul 16 21:53:54.296549 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.296519    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj_fe78c421-68ae-4b06-983c-f3ba2bd24747/util/0.log"
Jul 16 21:53:54.303407 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.303375    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj_fe78c421-68ae-4b06-983c-f3ba2bd24747/pull/0.log"
Jul 16 21:53:54.310804 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.310778    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj_fe78c421-68ae-4b06-983c-f3ba2bd24747/extract/0.log"
Jul 16 21:53:54.419403 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.419372    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw_bd963ca4-8972-4a62-840c-d4cbdaa88cf7/pull/0.log"
Jul 16 21:53:54.426103 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.426033    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw_bd963ca4-8972-4a62-840c-d4cbdaa88cf7/extract/0.log"
Jul 16 21:53:54.432877 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.432851    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw_bd963ca4-8972-4a62-840c-d4cbdaa88cf7/util/0.log"
Jul 16 21:53:54.542646 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.542619    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h_61654333-ca77-4a1e-822f-8a10d1b3ad73/extract/0.log"
Jul 16 21:53:54.550068 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.550044    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h_61654333-ca77-4a1e-822f-8a10d1b3ad73/util/0.log"
Jul 16 21:53:54.557409 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.557385    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h_61654333-ca77-4a1e-822f-8a10d1b3ad73/pull/0.log"
Jul 16 21:53:54.695180 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695088    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"]
Jul 16 21:53:54.695576 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695560    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy"
Jul 16 21:53:54.695627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695577    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy"
Jul 16 21:53:54.695627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695590    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy"
Jul 16 21:53:54.695627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695596    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy"
Jul 16 21:53:54.695627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695604    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:53:54.695627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695609    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:53:54.695627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695614    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:53:54.695627 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695621    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:53:54.695846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695681    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:53:54.695846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695689    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:53:54.695846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695697    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="123a1044-84b4-4721-86d1-b96696075576" containerName="revoke-keys"
Jul 16 21:53:54.695846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695703    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy"
Jul 16 21:53:54.695846 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.695709    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="92dcdb0f-9eac-4bfb-95e4-76bcb3327e05" containerName="istio-proxy"
Jul 16 21:53:54.700097 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.700079    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"
Jul 16 21:53:54.703275 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.703251    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:53:54.706610 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.706585    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"]
Jul 16 21:53:54.781466 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.781441    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4bgf5_38bd0e8d-85bf-489f-96e8-1b4685a8ebe7/manager/0.log"
Jul 16 21:53:54.851212 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.851174    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab66fb80-64c8-450c-9d34-a8139a22792f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt\" (UID: \"ab66fb80-64c8-450c-9d34-a8139a22792f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"
Jul 16 21:53:54.952555 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.952468    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab66fb80-64c8-450c-9d34-a8139a22792f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt\" (UID: \"ab66fb80-64c8-450c-9d34-a8139a22792f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"
Jul 16 21:53:54.953092 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:54.953069    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab66fb80-64c8-450c-9d34-a8139a22792f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt\" (UID: \"ab66fb80-64c8-450c-9d34-a8139a22792f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"
Jul 16 21:53:55.011506 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:55.011468    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"
Jul 16 21:53:55.151411 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:55.151384    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"]
Jul 16 21:53:55.153899 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:53:55.153869    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab66fb80_64c8_450c_9d34_a8139a22792f.slice/crio-b1658156f845acc7649f3bc138fb519826fa531fd900fed8c799ec5f955f16c2 WatchSource:0}: Error finding container b1658156f845acc7649f3bc138fb519826fa531fd900fed8c799ec5f955f16c2: Status 404 returned error can't find the container with id b1658156f845acc7649f3bc138fb519826fa531fd900fed8c799ec5f955f16c2
Jul 16 21:53:55.816601 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:55.816576    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4q9wpq_b4894a3a-2c73-44cf-8995-a78743186669/istio-proxy/0.log"
Jul 16 21:53:55.931449 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:55.931420    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hjsck_4cdbd363-83f5-4337-b1d3-8b2f5617cd2e/discovery/0.log"
Jul 16 21:53:56.043040 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:56.043008    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-796f7d4c6-snm79_0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf/kube-auth-proxy/0.log"
Jul 16 21:53:56.057521 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:56.057415    2572 generic.go:356] "Generic (PLEG): container finished" podID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerID="fd3d12f56e11a0944cbea5284620503a09fe62dc5227a45d06523d2cb64e35b0" exitCode=6
Jul 16 21:53:56.057521 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:56.057495    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt" event={"ID":"ab66fb80-64c8-450c-9d34-a8139a22792f","Type":"ContainerDied","Data":"fd3d12f56e11a0944cbea5284620503a09fe62dc5227a45d06523d2cb64e35b0"}
Jul 16 21:53:56.057736 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:56.057539    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt" event={"ID":"ab66fb80-64c8-450c-9d34-a8139a22792f","Type":"ContainerStarted","Data":"b1658156f845acc7649f3bc138fb519826fa531fd900fed8c799ec5f955f16c2"}
Jul 16 21:53:56.057736 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:56.057683    2572 scope.go:117] "RemoveContainer" containerID="fd3d12f56e11a0944cbea5284620503a09fe62dc5227a45d06523d2cb64e35b0"
Jul 16 21:53:56.289156 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:56.289129    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xbb86_cfa0262c-6293-4621-81bb-fe494452904d/istio-proxy/1.log"
Jul 16 21:53:56.979175 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:56.979092    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-cqd8n_21e2154d-d9d3-4a27-aa99-ce472878334d/main/0.log"
Jul 16 21:53:57.065030 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:57.064995    2572 generic.go:356] "Generic (PLEG): container finished" podID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerID="7957bd4f60eaf049ed1f2ee5b3c2b20a50f8f1cd12229eb2758ed356b08504f5" exitCode=6
Jul 16 21:53:57.065202 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:57.065090    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt" event={"ID":"ab66fb80-64c8-450c-9d34-a8139a22792f","Type":"ContainerDied","Data":"7957bd4f60eaf049ed1f2ee5b3c2b20a50f8f1cd12229eb2758ed356b08504f5"}
Jul 16 21:53:57.065202 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:57.065149    2572 scope.go:117] "RemoveContainer" containerID="fd3d12f56e11a0944cbea5284620503a09fe62dc5227a45d06523d2cb64e35b0"
Jul 16 21:53:57.065401 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:57.065384    2572 scope.go:117] "RemoveContainer" containerID="7957bd4f60eaf049ed1f2ee5b3c2b20a50f8f1cd12229eb2758ed356b08504f5"
Jul 16 21:53:57.065664 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:57.065639    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt_odh-ai-gateway-infra(ab66fb80-64c8-450c-9d34-a8139a22792f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f"
Jul 16 21:53:57.090515 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:57.090486    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8vmdh_70730c39-d907-4f2c-8c20-81d2dcd6fc8d/main/0.log"
Jul 16 21:53:58.072284 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:53:58.072259    2572 scope.go:117] "RemoveContainer" containerID="7957bd4f60eaf049ed1f2ee5b3c2b20a50f8f1cd12229eb2758ed356b08504f5"
Jul 16 21:53:58.072693 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:53:58.072470    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt_odh-ai-gateway-infra(ab66fb80-64c8-450c-9d34-a8139a22792f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f"
Jul 16 21:54:06.481962 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:06.481932    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zjlgl_61292e97-d6af-40aa-9f6b-c944ebe621b3/global-pull-secret-syncer/0.log"
Jul 16 21:54:06.673012 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:06.672982    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mnn2h_d7b58d65-9c6a-43c1-b913-981279a5ace0/konnectivity-agent/0.log"
Jul 16 21:54:06.697699 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:06.697668    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-67.ec2.internal_5112854b758de5dd94996dccba2987eb/haproxy/0.log"
Jul 16 21:54:11.679974 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.679946    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8_b6d8de27-f4d2-48c6-8e21-28883d5025cd/extract/0.log"
Jul 16 21:54:11.715613 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.715586    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8_b6d8de27-f4d2-48c6-8e21-28883d5025cd/util/0.log"
Jul 16 21:54:11.742741 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.742714    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cwsz8_b6d8de27-f4d2-48c6-8e21-28883d5025cd/pull/0.log"
Jul 16 21:54:11.774916 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.774882    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj_fe78c421-68ae-4b06-983c-f3ba2bd24747/extract/0.log"
Jul 16 21:54:11.811125 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.811090    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj_fe78c421-68ae-4b06-983c-f3ba2bd24747/util/0.log"
Jul 16 21:54:11.845989 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.845965    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c2zxj_fe78c421-68ae-4b06-983c-f3ba2bd24747/pull/0.log"
Jul 16 21:54:11.884626 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.884601    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw_bd963ca4-8972-4a62-840c-d4cbdaa88cf7/extract/0.log"
Jul 16 21:54:11.915397 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.915343    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw_bd963ca4-8972-4a62-840c-d4cbdaa88cf7/util/0.log"
Jul 16 21:54:11.940868 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.940792    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vx7sw_bd963ca4-8972-4a62-840c-d4cbdaa88cf7/pull/0.log"
Jul 16 21:54:11.973773 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:11.973744    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h_61654333-ca77-4a1e-822f-8a10d1b3ad73/extract/0.log"
Jul 16 21:54:12.007700 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:12.007675    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h_61654333-ca77-4a1e-822f-8a10d1b3ad73/util/0.log"
Jul 16 21:54:12.039510 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:12.039480    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12zc8h_61654333-ca77-4a1e-822f-8a10d1b3ad73/pull/0.log"
Jul 16 21:54:12.370966 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:12.370936    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4bgf5_38bd0e8d-85bf-489f-96e8-1b4685a8ebe7/manager/0.log"
Jul 16 21:54:13.244196 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:13.244168    2572 scope.go:117] "RemoveContainer" containerID="7957bd4f60eaf049ed1f2ee5b3c2b20a50f8f1cd12229eb2758ed356b08504f5"
Jul 16 21:54:14.161401 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:14.161298    2572 generic.go:356] "Generic (PLEG): container finished" podID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerID="ba649f54e19865c102537386905e9f8cc1830cc4de1601695dc60437050488ef" exitCode=6
Jul 16 21:54:14.161401 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:14.161384    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt" event={"ID":"ab66fb80-64c8-450c-9d34-a8139a22792f","Type":"ContainerDied","Data":"ba649f54e19865c102537386905e9f8cc1830cc4de1601695dc60437050488ef"}
Jul 16 21:54:14.161593 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:14.161444    2572 scope.go:117] "RemoveContainer" containerID="7957bd4f60eaf049ed1f2ee5b3c2b20a50f8f1cd12229eb2758ed356b08504f5"
Jul 16 21:54:14.161749 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:14.161729    2572 scope.go:117] "RemoveContainer" containerID="ba649f54e19865c102537386905e9f8cc1830cc4de1601695dc60437050488ef"
Jul 16 21:54:14.161977 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:54:14.161959    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt_odh-ai-gateway-infra(ab66fb80-64c8-450c-9d34-a8139a22792f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f"
Jul 16 21:54:14.858532 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:14.858497    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qjfdp_d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002/kube-state-metrics/0.log"
Jul 16 21:54:14.891159 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:14.891135    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qjfdp_d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002/kube-rbac-proxy-main/0.log"
Jul 16 21:54:14.917469 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:14.917436    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-qjfdp_d60cacca-9aa4-4c3d-b4ab-4c34c2dc3002/kube-rbac-proxy-self/0.log"
Jul 16 21:54:14.948653 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:14.948626    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-64cd49d94d-rk7lx_5996081f-4f04-4cf0-819f-ab4e356b2163/metrics-server/0.log"
Jul 16 21:54:15.194474 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.194385    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"]
Jul 16 21:54:15.306514 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.306486    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nrk4v_d5468131-84f7-47ef-be17-47499500f8a6/node-exporter/0.log"
Jul 16 21:54:15.333755 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.333730    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"
Jul 16 21:54:15.358424 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.358399    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nrk4v_d5468131-84f7-47ef-be17-47499500f8a6/kube-rbac-proxy/0.log"
Jul 16 21:54:15.396268 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.396240    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nrk4v_d5468131-84f7-47ef-be17-47499500f8a6/init-textfile/0.log"
Jul 16 21:54:15.432328 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.432298    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-26pzx_9dec2bf2-90be-4d7c-9263-7ec2db3f17bd/kube-rbac-proxy-main/0.log"
Jul 16 21:54:15.440298 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.440279    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab66fb80-64c8-450c-9d34-a8139a22792f-maas-api-service-ca\") pod \"ab66fb80-64c8-450c-9d34-a8139a22792f\" (UID: \"ab66fb80-64c8-450c-9d34-a8139a22792f\") "
Jul 16 21:54:15.440642 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.440619    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab66fb80-64c8-450c-9d34-a8139a22792f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ab66fb80-64c8-450c-9d34-a8139a22792f" (UID: "ab66fb80-64c8-450c-9d34-a8139a22792f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:54:15.464090 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.464011    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-26pzx_9dec2bf2-90be-4d7c-9263-7ec2db3f17bd/kube-rbac-proxy-self/0.log"
Jul 16 21:54:15.498057 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.498028    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-26pzx_9dec2bf2-90be-4d7c-9263-7ec2db3f17bd/openshift-state-metrics/0.log"
Jul 16 21:54:15.541514 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.541476    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab66fb80-64c8-450c-9d34-a8139a22792f-maas-api-service-ca\") on node \"ip-10-0-133-67.ec2.internal\" DevicePath \"\""
Jul 16 21:54:15.740808 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.740721    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-bx9qf_183b5c61-663b-423c-8b1b-e3c07131cb74/prometheus-operator/0.log"
Jul 16 21:54:15.764160 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.764136    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-bx9qf_183b5c61-663b-423c-8b1b-e3c07131cb74/kube-rbac-proxy/0.log"
Jul 16 21:54:15.829079 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.829023    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59f57d446c-l9rfc_8319506b-ee36-4500-8aa3-df405026e1bf/telemeter-client/0.log"
Jul 16 21:54:15.855809 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.855779    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59f57d446c-l9rfc_8319506b-ee36-4500-8aa3-df405026e1bf/reload/0.log"
Jul 16 21:54:15.885337 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.885307    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59f57d446c-l9rfc_8319506b-ee36-4500-8aa3-df405026e1bf/kube-rbac-proxy/0.log"
Jul 16 21:54:15.919897 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.919868    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6587bf557-82qg4_9f4701a8-47d7-4d26-adfa-1f748101d1d9/thanos-query/0.log"
Jul 16 21:54:15.945704 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.945679    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6587bf557-82qg4_9f4701a8-47d7-4d26-adfa-1f748101d1d9/kube-rbac-proxy-web/0.log"
Jul 16 21:54:15.971091 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:15.971062    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6587bf557-82qg4_9f4701a8-47d7-4d26-adfa-1f748101d1d9/kube-rbac-proxy/0.log"
Jul 16 21:54:16.008932 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.008851    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6587bf557-82qg4_9f4701a8-47d7-4d26-adfa-1f748101d1d9/prom-label-proxy/0.log"
Jul 16 21:54:16.043923 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.043896    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6587bf557-82qg4_9f4701a8-47d7-4d26-adfa-1f748101d1d9/kube-rbac-proxy-rules/0.log"
Jul 16 21:54:16.071185 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.071153    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6587bf557-82qg4_9f4701a8-47d7-4d26-adfa-1f748101d1d9/kube-rbac-proxy-metrics/0.log"
Jul 16 21:54:16.108074 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108039    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"]
Jul 16 21:54:16.108631 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108609    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:16.108631 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108631    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:16.108820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108648    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:54:16.108820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108656    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:54:16.108820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108670    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:16.108820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108679    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:16.108820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108774    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:16.108820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108789    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:16.108820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.108805    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6285aa9-5f19-49fc-8142-63f0b1faee34" containerName="revoke-keys"
Jul 16 21:54:16.112081 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.112058    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.114609 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.114588    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8lb8\"/\"kube-root-ca.crt\""
Jul 16 21:54:16.114702 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.114595    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8lb8\"/\"openshift-service-ca.crt\""
Jul 16 21:54:16.115366 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.115335    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g8lb8\"/\"default-dockercfg-5rzmt\""
Jul 16 21:54:16.124024 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.124000    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"]
Jul 16 21:54:16.147031 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.146995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-podres\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.147224 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.147044    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvdc9\" (UniqueName: \"kubernetes.io/projected/abb52d38-2aff-4aaa-81d3-263f10f9f84b-kube-api-access-gvdc9\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.147224 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.147077    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-sys\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.147224 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.147098    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-lib-modules\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.147224 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.147143    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-proc\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.176729 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.176701    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"
Jul 16 21:54:16.176899 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.176705    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt" event={"ID":"ab66fb80-64c8-450c-9d34-a8139a22792f","Type":"ContainerDied","Data":"b1658156f845acc7649f3bc138fb519826fa531fd900fed8c799ec5f955f16c2"}
Jul 16 21:54:16.176899 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.176821    2572 scope.go:117] "RemoveContainer" containerID="ba649f54e19865c102537386905e9f8cc1830cc4de1601695dc60437050488ef"
Jul 16 21:54:16.201883 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.201849    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"]
Jul 16 21:54:16.210768 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.210736    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7c0a35cd-b5cgt"]
Jul 16 21:54:16.247947 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.247914    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-podres\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.248120 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.247954    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvdc9\" (UniqueName: \"kubernetes.io/projected/abb52d38-2aff-4aaa-81d3-263f10f9f84b-kube-api-access-gvdc9\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.248120 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.247980    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-sys\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.248120 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.248032    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-lib-modules\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.248120 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.248038    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-sys\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.248120 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.248085    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-proc\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.248298 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.248122    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-proc\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.248298 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.248121    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-lib-modules\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.248298 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.248088    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/abb52d38-2aff-4aaa-81d3-263f10f9f84b-podres\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.257011 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.256982    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvdc9\" (UniqueName: \"kubernetes.io/projected/abb52d38-2aff-4aaa-81d3-263f10f9f84b-kube-api-access-gvdc9\") pod \"perf-node-gather-daemonset-5tkpn\" (UID: \"abb52d38-2aff-4aaa-81d3-263f10f9f84b\") " pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.423102 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.423060    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:16.562219 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:16.562191    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"]
Jul 16 21:54:16.563885 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:54:16.563851    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podabb52d38_2aff_4aaa_81d3_263f10f9f84b.slice/crio-39ac09cbaa784150c1d8f8fa4401bece684e6c54452030005cf3048d177c891f WatchSource:0}: Error finding container 39ac09cbaa784150c1d8f8fa4401bece684e6c54452030005cf3048d177c891f: Status 404 returned error can't find the container with id 39ac09cbaa784150c1d8f8fa4401bece684e6c54452030005cf3048d177c891f
Jul 16 21:54:17.184820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:17.184785    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn" event={"ID":"abb52d38-2aff-4aaa-81d3-263f10f9f84b","Type":"ContainerStarted","Data":"d6b31097fac2bc3307d4954ea7fece0950dc766bb966baa60f41496508a296d4"}
Jul 16 21:54:17.185182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:17.184836    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn" event={"ID":"abb52d38-2aff-4aaa-81d3-263f10f9f84b","Type":"ContainerStarted","Data":"39ac09cbaa784150c1d8f8fa4401bece684e6c54452030005cf3048d177c891f"}
Jul 16 21:54:17.185182 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:17.184920    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:17.245765 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:17.245711    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn" podStartSLOduration=1.245694042 podStartE2EDuration="1.245694042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:54:17.244252874 +0000 UTC m=+2322.755412609" watchObservedRunningTime="2026-07-16 21:54:17.245694042 +0000 UTC m=+2322.756853775"
Jul 16 21:54:17.250563 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:17.250535    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" path="/var/lib/kubelet/pods/ab66fb80-64c8-450c-9d34-a8139a22792f/volumes"
Jul 16 21:54:20.424343 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:20.424311    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pc7xc_db58efe2-e346-46d0-9827-86078b2c445a/dns/0.log"
Jul 16 21:54:20.450171 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:20.450148    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pc7xc_db58efe2-e346-46d0-9827-86078b2c445a/kube-rbac-proxy/0.log"
Jul 16 21:54:20.541429 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:20.541402    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zchn6_6e6e6771-9ce2-4db7-9d34-e9bfba359c4a/dns-node-resolver/0.log"
Jul 16 21:54:21.188142 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:21.188112    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qc6g5_d8572761-a70c-4a5c-a04a-2ffc3ccec888/node-ca/0.log"
Jul 16 21:54:22.241481 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:22.241447    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4q9wpq_b4894a3a-2c73-44cf-8995-a78743186669/istio-proxy/0.log"
Jul 16 21:54:22.316051 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:22.316017    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hjsck_4cdbd363-83f5-4337-b1d3-8b2f5617cd2e/discovery/0.log"
Jul 16 21:54:22.338424 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:22.338396    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-796f7d4c6-snm79_0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf/kube-auth-proxy/0.log"
Jul 16 21:54:22.409812 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:22.409778    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xbb86_cfa0262c-6293-4621-81bb-fe494452904d/istio-proxy/0.log"
Jul 16 21:54:22.437501 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:22.437469    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xbb86_cfa0262c-6293-4621-81bb-fe494452904d/istio-proxy/1.log"
Jul 16 21:54:23.200475 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.200449    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g8lb8/perf-node-gather-daemonset-5tkpn"
Jul 16 21:54:23.366152 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.366126    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pkgpg_819d0098-4e1a-4594-be74-a9307021a5ce/serve-healthcheck-canary/0.log"
Jul 16 21:54:23.635510 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.635478    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92"]
Jul 16 21:54:23.635880 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.635868    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:23.635921 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.635881    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:23.635956 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.635946    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab66fb80-64c8-450c-9d34-a8139a22792f" containerName="revoke-keys"
Jul 16 21:54:23.639272 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.639253    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92"
Jul 16 21:54:23.643512 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.643490    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7bksf\""
Jul 16 21:54:23.649112 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.649082    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92"]
Jul 16 21:54:23.721574 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.721533    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5baacc71-2673-4553-a5f9-0b9e8b2c211d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92\" (UID: \"5baacc71-2673-4553-a5f9-0b9e8b2c211d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92"
Jul 16 21:54:23.822500 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.822465    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5baacc71-2673-4553-a5f9-0b9e8b2c211d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92\" (UID: \"5baacc71-2673-4553-a5f9-0b9e8b2c211d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92"
Jul 16 21:54:23.823096 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.823076    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5baacc71-2673-4553-a5f9-0b9e8b2c211d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92\" (UID: \"5baacc71-2673-4553-a5f9-0b9e8b2c211d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92"
Jul 16 21:54:23.951075 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.950983    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92"
Jul 16 21:54:23.969579 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:23.969554    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2r98p_72a05823-0e7c-4d2b-b6be-2a6972a2827d/kube-rbac-proxy/0.log"
Jul 16 21:54:24.005109 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:24.005066    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2r98p_72a05823-0e7c-4d2b-b6be-2a6972a2827d/exporter/0.log"
Jul 16 21:54:24.034464 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:24.034440    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2r98p_72a05823-0e7c-4d2b-b6be-2a6972a2827d/extractor/0.log"
Jul 16 21:54:24.092492 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:24.092464    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92"]
Jul 16 21:54:24.093930 ip-10-0-133-67 kubenswrapper[2572]: W0716 21:54:24.093903    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5baacc71_2673_4553_a5f9_0b9e8b2c211d.slice/crio-e550e2193dbc121c18b815c9aa52eaf225e7fca1beba55209f115436ef80ae66 WatchSource:0}: Error finding container e550e2193dbc121c18b815c9aa52eaf225e7fca1beba55209f115436ef80ae66: Status 404 returned error can't find the container with id e550e2193dbc121c18b815c9aa52eaf225e7fca1beba55209f115436ef80ae66
Jul 16 21:54:24.224317 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:24.224225    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92" event={"ID":"5baacc71-2673-4553-a5f9-0b9e8b2c211d","Type":"ContainerStarted","Data":"e550e2193dbc121c18b815c9aa52eaf225e7fca1beba55209f115436ef80ae66"}
Jul 16 21:54:25.231688 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:25.231655    2572 generic.go:356] "Generic (PLEG): container finished" podID="5baacc71-2673-4553-a5f9-0b9e8b2c211d" containerID="0e715ef64a4dfb0d205b3c574142c31a1bd50ee08a9b8ff3717496502a0cf680" exitCode=6
Jul 16 21:54:25.232075 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:25.231739    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92" event={"ID":"5baacc71-2673-4553-a5f9-0b9e8b2c211d","Type":"ContainerDied","Data":"0e715ef64a4dfb0d205b3c574142c31a1bd50ee08a9b8ff3717496502a0cf680"}
Jul 16 21:54:25.232075 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:25.231899    2572 scope.go:117] "RemoveContainer" containerID="0e715ef64a4dfb0d205b3c574142c31a1bd50ee08a9b8ff3717496502a0cf680"
Jul 16 21:54:26.239652 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:26.239558    2572 generic.go:356] "Generic (PLEG): container finished" podID="5baacc71-2673-4553-a5f9-0b9e8b2c211d" containerID="93b27375217c29e6d34d3b894d70111403887b2ced46b4a2666322a61d8a284e" exitCode=6
Jul 16 21:54:26.240093 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:26.239649    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92" event={"ID":"5baacc71-2673-4553-a5f9-0b9e8b2c211d","Type":"ContainerDied","Data":"93b27375217c29e6d34d3b894d70111403887b2ced46b4a2666322a61d8a284e"}
Jul 16 21:54:26.240093 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:26.239709    2572 scope.go:117] "RemoveContainer" containerID="0e715ef64a4dfb0d205b3c574142c31a1bd50ee08a9b8ff3717496502a0cf680"
Jul 16 21:54:26.240093 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:26.239968    2572 scope.go:117] "RemoveContainer" containerID="93b27375217c29e6d34d3b894d70111403887b2ced46b4a2666322a61d8a284e"
Jul 16 21:54:26.240216 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:54:26.240178    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92_odh-ai-gateway-infra(5baacc71-2673-4553-a5f9-0b9e8b2c211d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92" podUID="5baacc71-2673-4553-a5f9-0b9e8b2c211d"
Jul 16 21:54:26.990410 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:26.990380    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f4k5g_82f32722-26a1-4261-a07d-a07ead89c7c5/manager/0.log"
Jul 16 21:54:27.078865 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.078828    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-dcswj_d7626221-53db-424f-abfb-958a90cc6f03/manager/0.log"
Jul 16 21:54:27.155579 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.155540    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5659cfddb-8rcp4_a49f83a4-d0d1-4892-a665-1fd5bdbfaf4a/manager/0.log"
Jul 16 21:54:27.185832 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.185803    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-n4s9m_bbee227a-765d-4d7d-87fd-ccad2a11870e/server/0.log"
Jul 16 21:54:27.220667 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.220638    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4q9wpq_b4894a3a-2c73-44cf-8995-a78743186669/istio-proxy/0.log"
Jul 16 21:54:27.253181 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.253089    2572 scope.go:117] "RemoveContainer" containerID="93b27375217c29e6d34d3b894d70111403887b2ced46b4a2666322a61d8a284e"
Jul 16 21:54:27.254209 ip-10-0-133-67 kubenswrapper[2572]: E0716 21:54:27.254174    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92_odh-ai-gateway-infra(5baacc71-2673-4553-a5f9-0b9e8b2c211d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f6d0b514-zbj92" podUID="5baacc71-2673-4553-a5f9-0b9e8b2c211d"
Jul 16 21:54:27.270754 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.270724    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-96f2w_c30d7193-e264-40d2-959e-7997f27c9760/manager/0.log"
Jul 16 21:54:27.296255 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.296220    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hjsck_4cdbd363-83f5-4337-b1d3-8b2f5617cd2e/discovery/0.log"
Jul 16 21:54:27.341996 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.341971    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-796f7d4c6-snm79_0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf/kube-auth-proxy/0.log"
Jul 16 21:54:27.388216 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.388187    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xbb86_cfa0262c-6293-4621-81bb-fe494452904d/istio-proxy/0.log"
Jul 16 21:54:27.415761 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.415726    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xbb86_cfa0262c-6293-4621-81bb-fe494452904d/istio-proxy/1.log"
Jul 16 21:54:27.441976 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.441956    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4b6cd7d5-947rd_bb765821-10ed-48ae-8b06-2472a6b9aea9/manager/0.log"
Jul 16 21:54:27.479405 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:27.479379    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c4b6cd7d5-947rd_bb765821-10ed-48ae-8b06-2472a6b9aea9/copy-manifests/0.log"
Jul 16 21:54:28.642864 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:28.642833    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4q9wpq_b4894a3a-2c73-44cf-8995-a78743186669/istio-proxy/0.log"
Jul 16 21:54:28.700073 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:28.700035    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-hjsck_4cdbd363-83f5-4337-b1d3-8b2f5617cd2e/discovery/0.log"
Jul 16 21:54:28.724788 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:28.724761    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-796f7d4c6-snm79_0ac3ed4d-04a1-4a1c-89f6-3b7ced1d77cf/kube-auth-proxy/0.log"
Jul 16 21:54:28.766992 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:28.766959    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xbb86_cfa0262c-6293-4621-81bb-fe494452904d/istio-proxy/0.log"
Jul 16 21:54:28.800820 ip-10-0-133-67 kubenswrapper[2572]: I0716 21:54:28.800785    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xbb86_cfa0262c-6293-4621-81bb-fe494452904d/istio-proxy/1.log"