Jul 17 01:11:30.774105 ip-10-0-134-254 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 01:11:30.774118 ip-10-0-134-254 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 01:11:30.774127 ip-10-0-134-254 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 01:11:30.774499 ip-10-0-134-254 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 01:11:40.773813 ip-10-0-134-254 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 17 01:11:40.773826 ip-10-0-134-254 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 3de368b9bc34403a8c4de0c19f41ce06 --
Jul 17 01:13:12.363801 ip-10-0-134-254 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 01:13:12.817187 ip-10-0-134-254 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 01:13:12.817187 ip-10-0-134-254 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 01:13:12.817187 ip-10-0-134-254 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 01:13:12.817187 ip-10-0-134-254 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 01:13:12.817187 ip-10-0-134-254 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 01:13:12.818131 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.818019    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 01:13:12.824086 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824070    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 01:13:12.824086 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824086    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 01:13:12.824086 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824090    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824096    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824100    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824103    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824105    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824108    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824110    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824113    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824115    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824121    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824125    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824128    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824131    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824133    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824136    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824139    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824141    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824144    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824147    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 01:13:12.824190 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824150    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824154    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824158    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824160    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824163    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824165    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824168    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824170    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824173    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824176    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824178    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824181    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824183    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824186    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824194    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824197    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824200    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824203    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824205    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 01:13:12.824658 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824208    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824210    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824212    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824215    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824217    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824220    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824222    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824224    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824227    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824229    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824231    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824233    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824236    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824238    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824241    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824244    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824249    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824252    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824255    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 01:13:12.825147 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824258    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824261    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824263    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824267    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824273    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824276    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824278    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824280    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824283    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824286    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824289    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824291    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824294    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824296    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824298    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824301    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824303    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824306    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824308    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824311    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 01:13:12.825634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824313    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824315    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824319    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824322    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824325    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824327    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824330    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824332    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824336    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824338    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824341    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824344    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824346    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824349    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824351    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824354    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824356    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824359    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824361    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.824364    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 01:13:12.826326 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825073    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825079    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825083    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825085    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825090    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825093    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825095    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825098    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825100    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825102    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825105    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825108    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825110    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825115    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825118    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825121    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825124    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825127    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825130    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 01:13:12.827084 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825132    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825137    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825140    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825143    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825146    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825150    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825153    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825156    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825159    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825162    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825165    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825168    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825170    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825173    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825176    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825179    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825182    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825185    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825188    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 01:13:12.827596 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825190    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825193    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825195    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825198    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825201    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825203    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825206    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825209    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825211    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825214    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825216    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825219    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825221    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825224    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825227    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825230    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825232    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825234    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825237    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825240    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 01:13:12.828094 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825242    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825246    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825248    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825250    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825253    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825257    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825260    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825262    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825265    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825267    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825270    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825273    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825275    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825279    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825281    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825284    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825286    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825288    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825291    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825293    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 01:13:12.828634 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825295    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825298    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825300    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825302    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825304    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825308    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825312    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825314    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825317    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825319    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825321    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825324    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825326    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825329    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825331    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825333    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825336    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825338    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825341    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825343    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 01:13:12.829115 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.825345    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826183    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826195    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826200    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826204    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826214    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826218    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826224    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826230    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826234    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826239    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826243    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826247    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826251    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826256    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826261    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826264    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826270    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826279    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826283    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 01:13:12.829624 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826288    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826292    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826298    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826303    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826307    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826313    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826317    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826322    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826326    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826336    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826340    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826345    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826349    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826354    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826358    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826383    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826387    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826391    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826396    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 01:13:12.830108 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826400    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826404    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826409    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826417    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826421    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826426    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826430    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826437    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826443    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826448    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826453    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826458    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826462    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826467    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826471    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826477    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826505    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826511    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826515    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826519    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 01:13:12.830575 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826523    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826528    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826532    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826536    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826540    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826544    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826549    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826553    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826561    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826586    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826590    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826595    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826599    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826603    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826608    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826612    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826618    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826622    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826627    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 01:13:12.831050 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826632    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826646    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826652    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826657    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826662    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826666    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826671    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826676    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826681    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826685    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826689    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826693    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826698    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826706    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826710    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826714    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826718    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826722    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826727    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 01:13:12.831534 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826731    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826736    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.826740    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826890    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826941    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826951    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826958    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826967    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826972    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826984    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826991    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.826995    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827004    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827010    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827015    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827021    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827025    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827029    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827037    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827041    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827046    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827063    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827068    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827072    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 01:13:12.831985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827077    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827082    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827091    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827096    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827101    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827106    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827110    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827114    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827118    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827123    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827127    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827136    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827141    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827160    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827165    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827169    2582 flags.go:64] FLAG: --enable-server="true"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827174    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827189    2582 flags.go:64] FLAG: --event-burst="100"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827194    2582 flags.go:64] FLAG: --event-qps="50"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827202    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827208    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827212    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827219    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827223    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827228    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 01:13:12.832558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827233    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827259    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827264    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827283    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827395    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827401    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827405    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827410    2582 flags.go:64] FLAG: --feature-gates=""
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827418    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827421    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827424    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827427    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827430    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827433    2582 flags.go:64] FLAG: --help="false"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827435    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-134-254.ec2.internal"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827439    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827441    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827444    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827448    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827451    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827454    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827456    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827459    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 01:13:12.833150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827462    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827464    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827467    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827470    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827473    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827476    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827479    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827481    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827484    2582 flags.go:64] FLAG: --lock-file=""
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827504    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827508    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827511    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827517    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827521    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827523    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827526    2582 flags.go:64] FLAG: --logging-format="text"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827529    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827534    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827537    2582 flags.go:64] FLAG: --manifest-url=""
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827540    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827544    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827547    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827551    2582 flags.go:64] FLAG: --max-pods="110"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827554    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827557    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 01:13:12.833840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827560    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827563    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827566    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827569    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827572    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827585    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827588    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827591    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827594    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827597    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827603    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827606    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827609    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827612    2582 flags.go:64] FLAG: --port="10250"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827615    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827618    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08fc29dd87503f26f"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827621    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827624    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827626    2582 flags.go:64] FLAG: --register-node="true"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827629    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827633    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827635    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827638    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827640    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827645    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 01:13:12.834538 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827648    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827651    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827654    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827657    2582 flags.go:64] FLAG: --runonce="false"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827660    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827663    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827665    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827668    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827670    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827673    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827676    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827679    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827681    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827684    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827687    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827689    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827692    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827695    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827698    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827704    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827706    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827709    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827714    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827716    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827719    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 01:13:12.835148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827721    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827724    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827727    2582 flags.go:64] FLAG: --v="2"
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827731    2582 flags.go:64] FLAG: --version="false"
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827734    2582 flags.go:64] FLAG: --vmodule=""
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827738    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.827743    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827896    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827901    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827904    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827907    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827910    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827912    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827915    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827918    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827920    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827923    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827925    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827928    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827930    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827933    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 01:13:12.835781 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827935    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827938    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827940    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827943    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827946    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827949    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827952    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827954    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827957    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827959    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827962    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827965    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827967    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827969    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827972    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827974    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827977    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827986    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827989    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827991    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 01:13:12.836278 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827994    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827996    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.827999    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828001    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828004    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828006    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828009    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828011    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828014    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828016    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828018    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828021    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828024    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828028    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828031    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828034    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828037    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828041    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828044    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 01:13:12.836782 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828046    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828049    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828052    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828055    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828057    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828060    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828074    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828078    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828081    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828084    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828089    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828098    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828101    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828103    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828106    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828108    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828111    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828113    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828116    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 01:13:12.837279 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828118    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828121    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828124    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828126    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828129    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828131    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828133    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828136    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828139    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828141    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828143    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828147    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828149    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828152    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828154    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828157    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828159    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828162    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828164    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828166    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 01:13:12.837777 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828170    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828173    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828176    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828180    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828183    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828193    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.828196    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.828868    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.836813    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.836830    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.836846    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.836852    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836915    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836920    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836924    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 01:13:12.838275 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836927    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836930    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836932    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836935    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836937    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836940    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836943    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836946    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836948    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836951    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836954    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836957    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836959    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836962    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836964    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836967    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836969    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836971    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836975    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836978    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 01:13:12.838672 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836981    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836983    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836986    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836988    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836991    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836993    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.836998    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837000    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837002    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837005    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837007    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837010    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837012    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837014    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837017    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837019    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837022    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837024    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837027    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 01:13:12.839150 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837031    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837034    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837037    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837040    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837043    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837046    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837048    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837051    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837054    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837056    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837060    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837063    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837065    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837068    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837070    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837072    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837075    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837077    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837079    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837082    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 01:13:12.839627 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837086    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837089    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837091    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837094    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837096    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837098    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837101    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837104    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837106    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837109    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837112    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837114    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837117    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837119    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837122    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837124    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837126    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837129    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837131    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837134    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 01:13:12.840104 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837136    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837139    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837141    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837144    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837146    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837148    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837151    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837154    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837156    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837159    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837161    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837164    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837167    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837170    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837172    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837176    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 01:13:12.840644 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837179    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.837184    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837303    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837308    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837312    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837316    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837319    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837321    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837324    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837327    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837329    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837332    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837334    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837337    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837339    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 01:13:12.841029 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837342    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837344    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837347    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837349    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837351    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837354    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837356    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837358    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837362    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837365    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837368    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837371    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837374    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837377    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837379    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837382    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837384    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837387    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837389    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 01:13:12.841384 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837392    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837394    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837396    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837399    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837401    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837404    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837406    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837408    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837411    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837413    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837416    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837419    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837422    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837425    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837428    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837431    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837433    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837436    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837439    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837441    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 01:13:12.841862 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837444    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837446    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837449    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837451    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837454    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837457    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837459    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837461    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837463    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837466    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837469    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837471    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837474    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837476    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837478    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837481    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837483    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837508    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837511    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 01:13:12.842361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837513    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837516    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837519    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837521    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837524    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837526    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837528    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837531    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837534    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837536    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837539    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837541    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837543    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837546    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837549    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837551    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837553    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837556    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837558    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837561    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 01:13:12.842826 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837563    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837565    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837568    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837571    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837574    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837576    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837579    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:12.837581    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.837586    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.837715    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.840019    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.841644    2582 server.go:1019] "Starting client certificate rotation"
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.841755    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 01:13:12.843302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.841789    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 01:13:12.844562 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.844547    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 01:13:12.846019 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.845993    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 01:13:12.854855 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.854836    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 17 01:13:12.859340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.859317    2582 log.go:25] "Validated CRI v1 image API"
Jul 17 01:13:12.860284 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.860250    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 01:13:12.862953 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.862932    2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 bf7d9ef9-03b6-4d1f-994d-2e4c0b111640:/dev/nvme0n1p3 ca041467-9257-4380-8eca-b409459cf2bb:/dev/nvme0n1p4]
Jul 17 01:13:12.863044 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.862952    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 01:13:12.869477 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.869362    2582 manager.go:216] Machine: {Timestamp:2026-07-17 01:13:12.867897779 +0000 UTC m=+0.371583634 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111772 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d3c74528d2c59abdc7e807711aa6f SystemUUID:ec2d3c74-528d-2c59-abdc-7e807711aa6f BootID:3de368b9-bc34-403a-8c4d-e0c19f41ce06 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4b:64:57:eb:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4b:64:57:eb:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:8f:83:06:3b:93 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 01:13:12.869477 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.869468    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 01:13:12.869628 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.869572    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 01:13:12.871111 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.871093    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 01:13:12.873682 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.873656    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 01:13:12.873814 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.873685    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-254.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 01:13:12.874512 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.874503    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 01:13:12.874543 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.874513    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 01:13:12.874543 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.874521    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 01:13:12.875590 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.875579    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 01:13:12.877532 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.877523    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 01:13:12.880945 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.880934    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 01:13:12.881092 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.881083    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 01:13:12.881122 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.881113    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 01:13:12.881149 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.881130    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 01:13:12.881149 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.881144    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 01:13:12.881204 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.881153    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 17 01:13:12.881204 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.881162    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 01:13:12.881514 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.881502    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 01:13:12.881559 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.881521    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 01:13:12.883855 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.883837    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 01:13:12.886015 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.886002    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 01:13:12.886079 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.886032    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 01:13:12.887513 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887500    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 01:13:12.887551 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887518    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 01:13:12.887551 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887524    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 01:13:12.887551 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887530    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 01:13:12.887551 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887539    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 01:13:12.887551 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887545    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 01:13:12.887551 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887553    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 01:13:12.887746 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887558    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 01:13:12.887746 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887565    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 01:13:12.887746 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887570    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 01:13:12.887746 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887578    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 01:13:12.887746 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.887584    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 01:13:12.889268 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.889257    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 01:13:12.889268 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.889267    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 01:13:12.891812 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.891776    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 01:13:12.891812 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.891793    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-254.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 01:13:12.891922 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.891830    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-254.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 01:13:12.892535 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.892523    2582 server.go:1268] "Started kubelet"
Jul 17 01:13:12.892639 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.892599    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 01:13:12.893438 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.893385    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 01:13:12.893516 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.893467    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 01:13:12.894743 ip-10-0-134-254 systemd[1]: Started Kubernetes Kubelet.
Jul 17 01:13:12.895665 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.895613    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 01:13:12.896168 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.896151    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 01:13:12.896910 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.896883    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 01:13:12.901585 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.900467    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-254.ec2.internal.18c2eea48c90b24c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-254.ec2.internal,UID:ip-10-0-134-254.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-254.ec2.internal,},FirstTimestamp:2026-07-17 01:13:12.892478028 +0000 UTC m=+0.396163881,LastTimestamp:2026-07-17 01:13:12.892478028 +0000 UTC m=+0.396163881,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-254.ec2.internal,}"
Jul 17 01:13:12.905281 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.905242    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 01:13:12.905376 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.905263    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 01:13:12.905456 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.905439    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:12.906352 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906338    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 01:13:12.906416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906363    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 01:13:12.906416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906402    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 01:13:12.906416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906410    2582 factory.go:55] Registering systemd factory
Jul 17 01:13:12.906562 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906442    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 17 01:13:12.906562 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906533    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 01:13:12.906562 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906542    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 01:13:12.906808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906792    2582 factory.go:153] Registering CRI-O factory
Jul 17 01:13:12.906808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906812    2582 factory.go:223] Registration of the crio container factory successfully
Jul 17 01:13:12.907085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906859    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 01:13:12.907085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906883    2582 factory.go:103] Registering Raw factory
Jul 17 01:13:12.907085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.906917    2582 manager.go:1196] Started watching for new ooms in manager
Jul 17 01:13:12.907323 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.907296    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-254.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 01:13:12.907810 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.907790    2582 manager.go:349] Starting recovery of all containers
Jul 17 01:13:12.915643 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.915617    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p9wxv"
Jul 17 01:13:12.917038 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.916968    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 01:13:12.920750 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.920733    2582 manager.go:354] Recovery completed
Jul 17 01:13:12.922673 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.922652    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p9wxv"
Jul 17 01:13:12.925028 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.925012    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 01:13:12.926813 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.926798    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 01:13:12.926900 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.926837    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 01:13:12.926900 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.926853    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientPID"
Jul 17 01:13:12.927455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.927439    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 01:13:12.927455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.927451    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 01:13:12.927581 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.927464    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 01:13:12.927581 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.927475    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 01:13:12.928155 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.928141    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 01:13:12.928216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.928170    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 01:13:12.928216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.928185    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientPID"
Jul 17 01:13:12.928862 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.928851    2582 policy_none.go:49] "None policy: Start"
Jul 17 01:13:12.928909 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.928866    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 01:13:12.928909 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.928876    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 01:13:12.929765 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.929755    2582 policy_none.go:47] "Start"
Jul 17 01:13:12.969243 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.969092    2582 manager.go:337] "Starting Device Plugin manager"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.969288    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.969304    2582 server.go:87] "Starting device plugin registration server"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.969643    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.969659    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.969758    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.969829    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.969836    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.970425    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.970475    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.982656    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.983988    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.984028    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.984048    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 01:13:12.984621 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:12.984079    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 01:13:12.986118 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:12.986038    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 01:13:13.070855 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.070784    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 01:13:13.071807 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.071788    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 01:13:13.071878 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.071822    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 01:13:13.071878 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.071833    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientPID"
Jul 17 01:13:13.071878 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.071856    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.078297 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.078277    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.078356 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.078300    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-254.ec2.internal\": node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.084152 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.084129    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"]
Jul 17 01:13:13.084250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.084186    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 01:13:13.084879 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.084862    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 01:13:13.084935 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.084888    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 01:13:13.084935 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.084898    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientPID"
Jul 17 01:13:13.085996 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.085984    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 01:13:13.086116 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.086101    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.086156 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.086138    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 01:13:13.086648 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.086629    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 01:13:13.086721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.086649    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 01:13:13.086721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.086661    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 01:13:13.086721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.086669    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 01:13:13.086721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.086676    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientPID"
Jul 17 01:13:13.086721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.086678    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientPID"
Jul 17 01:13:13.087741 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.087723    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.087830 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.087750    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 01:13:13.088340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.088327    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 01:13:13.088415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.088351    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 01:13:13.088415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.088365    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeHasSufficientPID"
Jul 17 01:13:13.102482 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.102466    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.107981 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.107958    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7f3251c29c2be847090021e12c41abdd-config\") pod \"kube-apiserver-proxy-ip-10-0-134-254.ec2.internal\" (UID: \"7f3251c29c2be847090021e12c41abdd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.108047 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.107995    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3060cc1582af17787ea2d991c3e8b345-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal\" (UID: \"3060cc1582af17787ea2d991c3e8b345\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.108047 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.108014    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3060cc1582af17787ea2d991c3e8b345-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal\" (UID: \"3060cc1582af17787ea2d991c3e8b345\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.115204 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.115175    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-254.ec2.internal\" not found" node="ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.119571 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.119556    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-254.ec2.internal\" not found" node="ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.203587 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.203557    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.208922 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.208905    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7f3251c29c2be847090021e12c41abdd-config\") pod \"kube-apiserver-proxy-ip-10-0-134-254.ec2.internal\" (UID: \"7f3251c29c2be847090021e12c41abdd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.208985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.208928    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3060cc1582af17787ea2d991c3e8b345-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal\" (UID: \"3060cc1582af17787ea2d991c3e8b345\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.208985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.208947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3060cc1582af17787ea2d991c3e8b345-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal\" (UID: \"3060cc1582af17787ea2d991c3e8b345\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.208985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.208980    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3060cc1582af17787ea2d991c3e8b345-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal\" (UID: \"3060cc1582af17787ea2d991c3e8b345\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.209092 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.209020    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3060cc1582af17787ea2d991c3e8b345-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal\" (UID: \"3060cc1582af17787ea2d991c3e8b345\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.209092 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.209018    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7f3251c29c2be847090021e12c41abdd-config\") pod \"kube-apiserver-proxy-ip-10-0-134-254.ec2.internal\" (UID: \"7f3251c29c2be847090021e12c41abdd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.304126 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.304082    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.404881 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.404820    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.419297 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.419272    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.424407 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.424393    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:13.505835 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.505800    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.606285 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.606248    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.706718 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.706636    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.807409 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.807371    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.840688 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.840668    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 01:13:13.841155 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.840824    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 01:13:13.908104 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:13.908077    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-254.ec2.internal\" not found"
Jul 17 01:13:13.927458 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.927420    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 01:08:12 +0000 UTC" deadline="2028-02-26 15:41:23.452441086 +0000 UTC"
Jul 17 01:13:13.927458 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.927456    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14150h28m9.52498739s"
Jul 17 01:13:13.990240 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:13.990160    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 01:13:14.006067 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.006038    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:14.015524 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.015476    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 01:13:14.017476 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.017460    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal"
Jul 17 01:13:14.028312 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.028288    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 01:13:14.136394 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:14.136357    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3060cc1582af17787ea2d991c3e8b345.slice/crio-79417f157e0170b5022ec79a85973d1a6efe3f477047dd81dd6a5ecc29f2979d WatchSource:0}: Error finding container 79417f157e0170b5022ec79a85973d1a6efe3f477047dd81dd6a5ecc29f2979d: Status 404 returned error can't find the container with id 79417f157e0170b5022ec79a85973d1a6efe3f477047dd81dd6a5ecc29f2979d
Jul 17 01:13:14.136946 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:14.136913    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f3251c29c2be847090021e12c41abdd.slice/crio-93cfdfbdba05a150fda74a23c2d4f3014859a27a1c091da7d0c4f72698d79997 WatchSource:0}: Error finding container 93cfdfbdba05a150fda74a23c2d4f3014859a27a1c091da7d0c4f72698d79997: Status 404 returned error can't find the container with id 93cfdfbdba05a150fda74a23c2d4f3014859a27a1c091da7d0c4f72698d79997
Jul 17 01:13:14.140690 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.140660    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 01:13:14.378653 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.378571    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 01:13:14.403981 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.403955    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 01:13:14.882743 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.882709    2582 apiserver.go:52] "Watching apiserver"
Jul 17 01:13:14.891900 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.891873    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 01:13:14.892264 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.892234    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal","openshift-multus/multus-blhf9","openshift-ovn-kubernetes/ovnkube-node-s6hjc","kube-system/konnectivity-agent-jlch5","kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal","openshift-dns/node-resolver-8g97m","openshift-image-registry/node-ca-5bmsd","openshift-multus/multus-additional-cni-plugins-pxbh2","openshift-multus/network-metrics-daemon-nbf9g","openshift-network-diagnostics/network-check-target-sshtq","openshift-network-operator/iptables-alerter-j5p9j","kube-system/global-pull-secret-syncer-flbd5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r","openshift-cluster-node-tuning-operator/tuned-9xvcr"]
Jul 17 01:13:14.893730 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.893695    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.895936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.895426    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.896607 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.896311    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 01:13:14.896607 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.896354    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 01:13:14.896607 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.896435    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 01:13:14.896798 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.896683    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:14.896798 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.896737    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bkbm8\""
Jul 17 01:13:14.896798 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.896749    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 01:13:14.897319 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.897301    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 01:13:14.897818 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.897643    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 01:13:14.897818 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.897774    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 01:13:14.897969 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.897952    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qxp8x\""
Jul 17 01:13:14.898161 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.898149    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 01:13:14.898283 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.898272    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 01:13:14.898817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.898635    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:14.898817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.898668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:14.898817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.898675    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 01:13:14.902706 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.901769    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.902706 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.901892    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 01:13:14.902706 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.902384    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 01:13:14.902706 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.902397    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8g8dr\""
Jul 17 01:13:14.903372 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.903331    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ccd26\""
Jul 17 01:13:14.904437 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904417    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 01:13:14.904541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904526    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 01:13:14.904599 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904573    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 01:13:14.904599 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904424    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 01:13:14.904758 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904743    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:14.904862 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:14.904807    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:14.904862 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904818    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 01:13:14.904862 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904845    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 01:13:14.905002 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904821    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 01:13:14.905002 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904895    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qwh4t\""
Jul 17 01:13:14.905002 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.904982    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fng76\""
Jul 17 01:13:14.905727 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.905709    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:14.905816 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:14.905773    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:14.905864 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.905826    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 01:13:14.907979 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.907786    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:14.909291 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.908982    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:14.909291 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:14.909048    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:14.909942 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.909735    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5mc4j\""
Jul 17 01:13:14.909942 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.909759    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 01:13:14.909942 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.909839    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 01:13:14.909942 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.909738    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 01:13:14.910179 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.910090    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:14.911439 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.911421    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:14.912033 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.912017    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 01:13:14.912333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.912316    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 01:13:14.912720 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.912701    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 01:13:14.912810 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.912724    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gs4w4\""
Jul 17 01:13:14.913407 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.913389    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 01:13:14.913523 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.913479    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 01:13:14.913711 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.913697    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kkd9c\""
Jul 17 01:13:14.917220 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.917201    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 01:13:14.918283 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-env-overrides\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918367 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918294    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e600bd66-c18f-4b53-8f5d-dfff57448245-hosts-file\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:14.918367 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918317    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-daemon-config\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.918367 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918356    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.918535 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-slash\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918535 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918406    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-etc-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918535 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918427    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-cni-bin\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918535 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-socket-dir-parent\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.918535 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918519    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-kubelet\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918718 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918548    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918718 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918580    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-ovn\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918718 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-netns\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.918718 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918624    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-run-ovn-kubernetes\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918718 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918673    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:14.918718 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918713    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-system-cni-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-cni-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-etc-kubernetes\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918798    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs5f2\" (UniqueName: \"kubernetes.io/projected/c75f5bce-b93f-4a73-8119-63546da6f555-kube-api-access-cs5f2\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918829    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e600bd66-c18f-4b53-8f5d-dfff57448245-tmp-dir\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918852    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e87c3885-4d60-419d-85f1-da2a84ae090a-iptables-alerter-script\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918874    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-systemd-units\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918895    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-cni-binary-copy\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e87c3885-4d60-419d-85f1-da2a84ae090a-host-slash\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:14.918941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-os-release\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918962    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-cni-binary-copy\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.918992    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-systemd\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919015    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-hostroot\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919035    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5f2c00d-8aa2-4181-99df-f099066c8a5b-agent-certs\") pod \"konnectivity-agent-jlch5\" (UID: \"a5f2c00d-8aa2-4181-99df-f099066c8a5b\") " pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919056    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-os-release\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-cni-bin\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919125    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-conf-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc6c8\" (UniqueName: \"kubernetes.io/projected/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-kube-api-access-dc6c8\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919185    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-cnibin\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-var-lib-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919281    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovnkube-config\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919312    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovnkube-script-lib\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzkqh\" (UniqueName: \"kubernetes.io/projected/e600bd66-c18f-4b53-8f5d-dfff57448245-kube-api-access-lzkqh\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919365    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtcvh\" (UniqueName: \"kubernetes.io/projected/ff931d44-0f32-4884-87a5-0d657a1afda9-kube-api-access-gtcvh\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919385    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-cnibin\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.919467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919405    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-multus-certs\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919446    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919474    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd9t4\" (UniqueName: \"kubernetes.io/projected/c8690b31-f48e-4eec-aafb-c91fb3cf442a-kube-api-access-pd9t4\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-node-log\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919550    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovn-node-metrics-cert\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919562    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-k8s-cni-cncf-io\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919575    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-kubelet\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919606    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff931d44-0f32-4884-87a5-0d657a1afda9-host\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919705    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff931d44-0f32-4884-87a5-0d657a1afda9-serviceca\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919741    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbrnx\" (UniqueName: \"kubernetes.io/projected/e87c3885-4d60-419d-85f1-da2a84ae090a-kube-api-access-hbrnx\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919758    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-cni-multus\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919772    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919816    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6xr9\" (UniqueName: \"kubernetes.io/projected/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-kube-api-access-f6xr9\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-cni-netd\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.920217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5f2c00d-8aa2-4181-99df-f099066c8a5b-konnectivity-ca\") pod \"konnectivity-agent-jlch5\" (UID: \"a5f2c00d-8aa2-4181-99df-f099066c8a5b\") " pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:14.920816 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-system-cni-dir\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:14.920816 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919931    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-run-netns\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.920816 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.919944    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-log-socket\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:14.935663 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.935640    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hm9lf"
Jul 17 01:13:14.942715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.942679    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hm9lf"
Jul 17 01:13:14.996322 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.996256    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal" event={"ID":"7f3251c29c2be847090021e12c41abdd","Type":"ContainerStarted","Data":"93cfdfbdba05a150fda74a23c2d4f3014859a27a1c091da7d0c4f72698d79997"}
Jul 17 01:13:14.997628 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:14.997595    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal" event={"ID":"3060cc1582af17787ea2d991c3e8b345","Type":"ContainerStarted","Data":"79417f157e0170b5022ec79a85973d1a6efe3f477047dd81dd6a5ecc29f2979d"}
Jul 17 01:13:15.007911 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.007883    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 01:13:15.020596 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020568    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-system-cni-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.020596 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-cni-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020616    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-etc-kubernetes\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs5f2\" (UniqueName: \"kubernetes.io/projected/c75f5bce-b93f-4a73-8119-63546da6f555-kube-api-access-cs5f2\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020666    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysctl-d\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020688    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-tuned\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020695    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-etc-kubernetes\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020704    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-system-cni-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020710    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e600bd66-c18f-4b53-8f5d-dfff57448245-tmp-dir\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020760    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e87c3885-4d60-419d-85f1-da2a84ae090a-iptables-alerter-script\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020815    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-socket-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020837    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysctl-conf\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020844    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-cni-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.020860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020862    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-systemd-units\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-cni-binary-copy\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020898    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020945    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysconfig\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020963    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e87c3885-4d60-419d-85f1-da2a84ae090a-host-slash\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-os-release\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.020994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-cni-binary-copy\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021008    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-device-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-systemd\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021037    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-hostroot\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021046    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e87c3885-4d60-419d-85f1-da2a84ae090a-host-slash\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5f2c00d-8aa2-4181-99df-f099066c8a5b-agent-certs\") pod \"konnectivity-agent-jlch5\" (UID: \"a5f2c00d-8aa2-4181-99df-f099066c8a5b\") " pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-os-release\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021118    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b09b456f-062e-4b27-b5cc-4cfaff662554-kubelet-config\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021144    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-systemd\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-cni-bin\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021187    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-conf-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.021449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc6c8\" (UniqueName: \"kubernetes.io/projected/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-kube-api-access-dc6c8\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-cnibin\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-var-lib-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021277    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovnkube-config\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021287    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021299    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovnkube-script-lib\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021303    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e87c3885-4d60-419d-85f1-da2a84ae090a-iptables-alerter-script\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021321    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzkqh\" (UniqueName: \"kubernetes.io/projected/e600bd66-c18f-4b53-8f5d-dfff57448245-kube-api-access-lzkqh\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcvh\" (UniqueName: \"kubernetes.io/projected/ff931d44-0f32-4884-87a5-0d657a1afda9-kube-api-access-gtcvh\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021371    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-cnibin\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-multus-certs\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021412    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021437    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-sys-fs\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021450    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-cni-binary-copy\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021466    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-modprobe-d\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021510    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd9t4\" (UniqueName: \"kubernetes.io/projected/c8690b31-f48e-4eec-aafb-c91fb3cf442a-kube-api-access-pd9t4\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-systemd-units\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-sys\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021574    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-cni-bin\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021613    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-var-lib-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021667    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-os-release\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021765    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-conf-dir\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021861    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-multus-certs\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021914    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-systemd\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021006    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e600bd66-c18f-4b53-8f5d-dfff57448245-tmp-dir\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.021995    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-hostroot\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022063    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-cni-binary-copy\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022071    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovnkube-script-lib\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022098    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-cnibin\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022126    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-host\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022151    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-registration-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022207    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpc8r\" (UniqueName: \"kubernetes.io/projected/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-kube-api-access-kpc8r\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022216    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovnkube-config\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022234    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-node-log\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.022936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022257    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovn-node-metrics-cert\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022263    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-cnibin\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-k8s-cni-cncf-io\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-kubelet\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022331    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-kubernetes\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-os-release\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022375    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-node-log\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022372    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-k8s-cni-cncf-io\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-kubelet\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022414    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff931d44-0f32-4884-87a5-0d657a1afda9-host\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022421    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff931d44-0f32-4884-87a5-0d657a1afda9-serviceca\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022464    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff931d44-0f32-4884-87a5-0d657a1afda9-host\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022518    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbrnx\" (UniqueName: \"kubernetes.io/projected/e87c3885-4d60-419d-85f1-da2a84ae090a-kube-api-access-hbrnx\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022557    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-cni-multus\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022660    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.023779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022688    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-var-lib-cni-multus\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6xr9\" (UniqueName: \"kubernetes.io/projected/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-kube-api-access-f6xr9\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff931d44-0f32-4884-87a5-0d657a1afda9-serviceca\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022904    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-cni-netd\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022945    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-cni-netd\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.022971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5f2c00d-8aa2-4181-99df-f099066c8a5b-konnectivity-ca\") pod \"konnectivity-agent-jlch5\" (UID: \"a5f2c00d-8aa2-4181-99df-f099066c8a5b\") " pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023074    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-system-cni-dir\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023100    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b09b456f-062e-4b27-b5cc-4cfaff662554-dbus\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-run\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023144    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-lib-modules\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023246    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023272    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c75f5bce-b93f-4a73-8119-63546da6f555-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-run-netns\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023364    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-log-socket\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023385    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-env-overrides\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023404    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e600bd66-c18f-4b53-8f5d-dfff57448245-hosts-file\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:15.024415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023363    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-system-cni-dir\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-daemon-config\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023553    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-run-netns\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023618    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e600bd66-c18f-4b53-8f5d-dfff57448245-hosts-file\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023639    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a5f2c00d-8aa2-4181-99df-f099066c8a5b-konnectivity-ca\") pod \"konnectivity-agent-jlch5\" (UID: \"a5f2c00d-8aa2-4181-99df-f099066c8a5b\") " pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023659    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-log-socket\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023699    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023723    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xwcq\" (UniqueName: \"kubernetes.io/projected/08257645-b30f-4cb5-836c-f89f162cf8c8-kube-api-access-5xwcq\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023738    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-slash\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023759    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-etc-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023773    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-cni-bin\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-socket-dir-parent\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023805    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-var-lib-kubelet\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023839    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-kubelet\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023843    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c8690b31-f48e-4eec-aafb-c91fb3cf442a-env-overrides\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023863    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023880    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-slash\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023884    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-ovn\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023908    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-etc-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023910    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-daemon-config\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023928    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-cni-bin\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023931    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-netns\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023956    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-multus-socket-dir-parent\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-run-ovn-kubernetes\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.023992    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.024000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c75f5bce-b93f-4a73-8119-63546da6f555-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.024009    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-tmp\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.024039    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-host-run-netns\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.024042    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-openvswitch\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.024071    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-run-ovn\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.024076    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-kubelet\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.024134    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8690b31-f48e-4eec-aafb-c91fb3cf442a-host-run-ovn-kubernetes\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.024169    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.024244    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs podName:d4845309-a57a-4d8d-8b9e-6a23aeaab9c8 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:15.524225653 +0000 UTC m=+3.027911505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs") pod "network-metrics-daemon-nbf9g" (UID: "d4845309-a57a-4d8d-8b9e-6a23aeaab9c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:15.025734 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.025570    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c8690b31-f48e-4eec-aafb-c91fb3cf442a-ovn-node-metrics-cert\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.026214 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.025611    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a5f2c00d-8aa2-4181-99df-f099066c8a5b-agent-certs\") pod \"konnectivity-agent-jlch5\" (UID: \"a5f2c00d-8aa2-4181-99df-f099066c8a5b\") " pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:15.026590 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.026569    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 01:13:15.026843 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.026598    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 01:13:15.026843 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.026611    2582 projected.go:196] Error preparing data for projected volume kube-api-access-jws6f for pod openshift-network-diagnostics/network-check-target-sshtq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:15.026843 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.026678    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f podName:625502d2-d5e4-4d07-b8ec-830465847680 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:15.526663102 +0000 UTC m=+3.030348961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jws6f" (UniqueName: "kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f") pod "network-check-target-sshtq" (UID: "625502d2-d5e4-4d07-b8ec-830465847680") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:15.027888 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.027867    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs5f2\" (UniqueName: \"kubernetes.io/projected/c75f5bce-b93f-4a73-8119-63546da6f555-kube-api-access-cs5f2\") pod \"multus-additional-cni-plugins-pxbh2\" (UID: \"c75f5bce-b93f-4a73-8119-63546da6f555\") " pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.028514 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.028477    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd9t4\" (UniqueName: \"kubernetes.io/projected/c8690b31-f48e-4eec-aafb-c91fb3cf442a-kube-api-access-pd9t4\") pod \"ovnkube-node-s6hjc\" (UID: \"c8690b31-f48e-4eec-aafb-c91fb3cf442a\") " pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.028609 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.028594    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzkqh\" (UniqueName: \"kubernetes.io/projected/e600bd66-c18f-4b53-8f5d-dfff57448245-kube-api-access-lzkqh\") pod \"node-resolver-8g97m\" (UID: \"e600bd66-c18f-4b53-8f5d-dfff57448245\") " pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:15.029381 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.029359    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc6c8\" (UniqueName: \"kubernetes.io/projected/db00c4f8-4c3f-4788-85a3-8fbd47cc557d-kube-api-access-dc6c8\") pod \"multus-blhf9\" (UID: \"db00c4f8-4c3f-4788-85a3-8fbd47cc557d\") " pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.029608 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.029591    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtcvh\" (UniqueName: \"kubernetes.io/projected/ff931d44-0f32-4884-87a5-0d657a1afda9-kube-api-access-gtcvh\") pod \"node-ca-5bmsd\" (UID: \"ff931d44-0f32-4884-87a5-0d657a1afda9\") " pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:15.032067 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.032049    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6xr9\" (UniqueName: \"kubernetes.io/projected/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-kube-api-access-f6xr9\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:15.034647 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.034631    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbrnx\" (UniqueName: \"kubernetes.io/projected/e87c3885-4d60-419d-85f1-da2a84ae090a-kube-api-access-hbrnx\") pod \"iptables-alerter-j5p9j\" (UID: \"e87c3885-4d60-419d-85f1-da2a84ae090a\") " pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:15.125110 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-kubernetes\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125282 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125126    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b09b456f-062e-4b27-b5cc-4cfaff662554-dbus\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.125282 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-run\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125282 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-lib-modules\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125282 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125189    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-kubernetes\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125282 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125197    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcq\" (UniqueName: \"kubernetes.io/projected/08257645-b30f-4cb5-836c-f89f162cf8c8-kube-api-access-5xwcq\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.125282 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125252    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-var-lib-kubelet\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125282 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125249    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-run\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125302    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-var-lib-kubelet\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125307    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b09b456f-062e-4b27-b5cc-4cfaff662554-dbus\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125353    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-tmp\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysctl-d\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125378    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-lib-modules\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125414    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-tuned\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125451    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125509    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-socket-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125534    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysctl-conf\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125509    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysctl-d\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125575    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.125583    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysconfig\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-device-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125632    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-socket-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.125636 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.125645    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret podName:b09b456f-062e-4b27-b5cc-4cfaff662554 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:15.625621553 +0000 UTC m=+3.129307392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret") pod "global-pull-secret-syncer-flbd5" (UID: "b09b456f-062e-4b27-b5cc-4cfaff662554") : object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125666    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-device-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125674    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysctl-conf\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125674    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125680    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b09b456f-062e-4b27-b5cc-4cfaff662554-kubelet-config\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125706    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-sysconfig\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125718    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b09b456f-062e-4b27-b5cc-4cfaff662554-kubelet-config\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-systemd\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125763    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-sys-fs\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125786    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-modprobe-d\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125808    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-sys\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-systemd\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125831    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-host\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125828    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-sys-fs\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125853    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-registration-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125881    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-modprobe-d\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125854    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-sys\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.126591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-host\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.127332 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125884    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.127332 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpc8r\" (UniqueName: \"kubernetes.io/projected/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-kube-api-access-kpc8r\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.127332 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.127332 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.125890    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/08257645-b30f-4cb5-836c-f89f162cf8c8-registration-dir\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.127816 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.127790    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-etc-tuned\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.127869 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.127830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-tmp\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.137144 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.137088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpc8r\" (UniqueName: \"kubernetes.io/projected/8fe448c4-0355-4d02-8e50-e82e0a9c7ac5-kube-api-access-kpc8r\") pod \"tuned-9xvcr\" (UID: \"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5\") " pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.137144 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.137133    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xwcq\" (UniqueName: \"kubernetes.io/projected/08257645-b30f-4cb5-836c-f89f162cf8c8-kube-api-access-5xwcq\") pod \"aws-ebs-csi-driver-node-djm8r\" (UID: \"08257645-b30f-4cb5-836c-f89f162cf8c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.205161 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.205119    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-blhf9"
Jul 17 01:13:15.215830 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.215805    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:15.224541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.224517    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:15.230807 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.230789    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5bmsd"
Jul 17 01:13:15.237440 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.237415    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8g97m"
Jul 17 01:13:15.247050 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.247024    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pxbh2"
Jul 17 01:13:15.254928 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.254906    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j5p9j"
Jul 17 01:13:15.261397 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.261375    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r"
Jul 17 01:13:15.268243 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.268224    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9xvcr"
Jul 17 01:13:15.285000 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.284973    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 01:13:15.529015 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.528924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:15.529015 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.529007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:15.529226 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.529085    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:15.529226 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.529151    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs podName:d4845309-a57a-4d8d-8b9e-6a23aeaab9c8 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:16.52913575 +0000 UTC m=+4.032821592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs") pod "network-metrics-daemon-nbf9g" (UID: "d4845309-a57a-4d8d-8b9e-6a23aeaab9c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:15.529226 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.529152    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 01:13:15.529226 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.529173    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 01:13:15.529226 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.529184    2582 projected.go:196] Error preparing data for projected volume kube-api-access-jws6f for pod openshift-network-diagnostics/network-check-target-sshtq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:15.529424 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.529252    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f podName:625502d2-d5e4-4d07-b8ec-830465847680 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:16.529236803 +0000 UTC m=+4.032922657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jws6f" (UniqueName: "kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f") pod "network-check-target-sshtq" (UID: "625502d2-d5e4-4d07-b8ec-830465847680") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:15.629766 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.629724    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:15.629980 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.629920    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:15.630057 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.630005    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret podName:b09b456f-062e-4b27-b5cc-4cfaff662554 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:16.62998318 +0000 UTC m=+4.133669040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret") pod "global-pull-secret-syncer-flbd5" (UID: "b09b456f-062e-4b27-b5cc-4cfaff662554") : object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:15.945988 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.945926    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 01:08:14 +0000 UTC" deadline="2027-12-18 04:05:50.252906597 +0000 UTC"
Jul 17 01:13:15.945988 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.945979    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12458h52m34.306930209s"
Jul 17 01:13:15.984413 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:15.984382    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:15.984581 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:15.984516    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:16.137046 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:16.137006    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff931d44_0f32_4884_87a5_0d657a1afda9.slice/crio-af255dd133072ac8cbdbfd14e5c74751ec64c2faf6912df162a3c6e2a3d03bbb WatchSource:0}: Error finding container af255dd133072ac8cbdbfd14e5c74751ec64c2faf6912df162a3c6e2a3d03bbb: Status 404 returned error can't find the container with id af255dd133072ac8cbdbfd14e5c74751ec64c2faf6912df162a3c6e2a3d03bbb
Jul 17 01:13:16.138157 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:16.138132    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8690b31_f48e_4eec_aafb_c91fb3cf442a.slice/crio-943d2a7da845c4943fda5edcdf22c261c7ae90dbbc1b840ca60303c4e3725ce6 WatchSource:0}: Error finding container 943d2a7da845c4943fda5edcdf22c261c7ae90dbbc1b840ca60303c4e3725ce6: Status 404 returned error can't find the container with id 943d2a7da845c4943fda5edcdf22c261c7ae90dbbc1b840ca60303c4e3725ce6
Jul 17 01:13:16.139156 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:16.139079    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe448c4_0355_4d02_8e50_e82e0a9c7ac5.slice/crio-176bd8fc3f2eea945a7682e4308cec00c8f7d8b2b5bcf3b90d4d200ad064a202 WatchSource:0}: Error finding container 176bd8fc3f2eea945a7682e4308cec00c8f7d8b2b5bcf3b90d4d200ad064a202: Status 404 returned error can't find the container with id 176bd8fc3f2eea945a7682e4308cec00c8f7d8b2b5bcf3b90d4d200ad064a202
Jul 17 01:13:16.143234 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:16.142184    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08257645_b30f_4cb5_836c_f89f162cf8c8.slice/crio-1dc42234c2926737497c0a08131bd9d67ea78ca71415444564defc10fef181f4 WatchSource:0}: Error finding container 1dc42234c2926737497c0a08131bd9d67ea78ca71415444564defc10fef181f4: Status 404 returned error can't find the container with id 1dc42234c2926737497c0a08131bd9d67ea78ca71415444564defc10fef181f4
Jul 17 01:13:16.144959 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:16.144594    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f2c00d_8aa2_4181_99df_f099066c8a5b.slice/crio-871f243e32fba66597d5d41e88645de87de1704994a5ef69c777089638336f11 WatchSource:0}: Error finding container 871f243e32fba66597d5d41e88645de87de1704994a5ef69c777089638336f11: Status 404 returned error can't find the container with id 871f243e32fba66597d5d41e88645de87de1704994a5ef69c777089638336f11
Jul 17 01:13:16.145548 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:16.145458    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode600bd66_c18f_4b53_8f5d_dfff57448245.slice/crio-1b0b7787b1df88f7109b3ee63bcd8127a92d9f014351ed3bb750a41f450db38c WatchSource:0}: Error finding container 1b0b7787b1df88f7109b3ee63bcd8127a92d9f014351ed3bb750a41f450db38c: Status 404 returned error can't find the container with id 1b0b7787b1df88f7109b3ee63bcd8127a92d9f014351ed3bb750a41f450db38c
Jul 17 01:13:16.147199 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:16.147055    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode87c3885_4d60_419d_85f1_da2a84ae090a.slice/crio-c0363a6d045dcbcee8e7e5e5070eb3627fa8ce308d77a83b25ec9ee0ea7f84f8 WatchSource:0}: Error finding container c0363a6d045dcbcee8e7e5e5070eb3627fa8ce308d77a83b25ec9ee0ea7f84f8: Status 404 returned error can't find the container with id c0363a6d045dcbcee8e7e5e5070eb3627fa8ce308d77a83b25ec9ee0ea7f84f8
Jul 17 01:13:16.148382 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:16.148359    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75f5bce_b93f_4a73_8119_63546da6f555.slice/crio-d79017609d470d22a4f21b254f9d4a753678401fde6703641ebb6d63fe07c051 WatchSource:0}: Error finding container d79017609d470d22a4f21b254f9d4a753678401fde6703641ebb6d63fe07c051: Status 404 returned error can't find the container with id d79017609d470d22a4f21b254f9d4a753678401fde6703641ebb6d63fe07c051
Jul 17 01:13:16.538631 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:16.538426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:16.538631 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.538617    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 01:13:16.538631 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.538641    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 01:13:16.538869 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.538652    2582 projected.go:196] Error preparing data for projected volume kube-api-access-jws6f for pod openshift-network-diagnostics/network-check-target-sshtq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:16.538869 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:16.538658    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:16.538869 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.538700    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f podName:625502d2-d5e4-4d07-b8ec-830465847680 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:18.538685021 +0000 UTC m=+6.042370860 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jws6f" (UniqueName: "kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f") pod "network-check-target-sshtq" (UID: "625502d2-d5e4-4d07-b8ec-830465847680") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:16.538869 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.538740    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:16.538869 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.538782    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs podName:d4845309-a57a-4d8d-8b9e-6a23aeaab9c8 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:18.538771659 +0000 UTC m=+6.042457499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs") pod "network-metrics-daemon-nbf9g" (UID: "d4845309-a57a-4d8d-8b9e-6a23aeaab9c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:16.639465 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:16.639382    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:16.639661 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.639514    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:16.639661 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.639583    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret podName:b09b456f-062e-4b27-b5cc-4cfaff662554 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:18.639565165 +0000 UTC m=+6.143251018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret") pod "global-pull-secret-syncer-flbd5" (UID: "b09b456f-062e-4b27-b5cc-4cfaff662554") : object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:16.946833 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:16.946742    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 01:08:14 +0000 UTC" deadline="2028-04-23 06:11:01.380544024 +0000 UTC"
Jul 17 01:13:16.946833 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:16.946786    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15508h57m44.433763522s"
Jul 17 01:13:16.987623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:16.987592    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:16.987769 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.987716    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:16.988118 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:16.988097    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:16.988229 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:16.988209    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:17.010444 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.010409    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8g97m" event={"ID":"e600bd66-c18f-4b53-8f5d-dfff57448245","Type":"ContainerStarted","Data":"1b0b7787b1df88f7109b3ee63bcd8127a92d9f014351ed3bb750a41f450db38c"}
Jul 17 01:13:17.015266 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.015236    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j5p9j" event={"ID":"e87c3885-4d60-419d-85f1-da2a84ae090a","Type":"ContainerStarted","Data":"c0363a6d045dcbcee8e7e5e5070eb3627fa8ce308d77a83b25ec9ee0ea7f84f8"}
Jul 17 01:13:17.017773 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.017744    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r" event={"ID":"08257645-b30f-4cb5-836c-f89f162cf8c8","Type":"ContainerStarted","Data":"1dc42234c2926737497c0a08131bd9d67ea78ca71415444564defc10fef181f4"}
Jul 17 01:13:17.019465 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.019437    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5bmsd" event={"ID":"ff931d44-0f32-4884-87a5-0d657a1afda9","Type":"ContainerStarted","Data":"af255dd133072ac8cbdbfd14e5c74751ec64c2faf6912df162a3c6e2a3d03bbb"}
Jul 17 01:13:17.023836 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.023810    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal" event={"ID":"7f3251c29c2be847090021e12c41abdd","Type":"ContainerStarted","Data":"9ad39a8c564af9eca513f4ff60c48ccd8fa88cf21b9951a0a7eed12638e240c8"}
Jul 17 01:13:17.026872 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.026846    2582 generic.go:356] "Generic (PLEG): container finished" podID="3060cc1582af17787ea2d991c3e8b345" containerID="3d2ccbeb267d9c713835d066458559d4bbf91a2557c0b52cc472faf76e83ee0b" exitCode=0
Jul 17 01:13:17.026970 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.026951    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal" event={"ID":"3060cc1582af17787ea2d991c3e8b345","Type":"ContainerDied","Data":"3d2ccbeb267d9c713835d066458559d4bbf91a2557c0b52cc472faf76e83ee0b"}
Jul 17 01:13:17.032999 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.032972    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" event={"ID":"c75f5bce-b93f-4a73-8119-63546da6f555","Type":"ContainerStarted","Data":"d79017609d470d22a4f21b254f9d4a753678401fde6703641ebb6d63fe07c051"}
Jul 17 01:13:17.039053 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.039027    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jlch5" event={"ID":"a5f2c00d-8aa2-4181-99df-f099066c8a5b","Type":"ContainerStarted","Data":"871f243e32fba66597d5d41e88645de87de1704994a5ef69c777089638336f11"}
Jul 17 01:13:17.046847 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.046561    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-blhf9" event={"ID":"db00c4f8-4c3f-4788-85a3-8fbd47cc557d","Type":"ContainerStarted","Data":"693bb802875558174291b4a290e507da519f8cfe2b78aec9671c20f71b58e91f"}
Jul 17 01:13:17.051119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.051039    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9xvcr" event={"ID":"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5","Type":"ContainerStarted","Data":"176bd8fc3f2eea945a7682e4308cec00c8f7d8b2b5bcf3b90d4d200ad064a202"}
Jul 17 01:13:17.052712 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.052672    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"943d2a7da845c4943fda5edcdf22c261c7ae90dbbc1b840ca60303c4e3725ce6"}
Jul 17 01:13:17.056178 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.056117    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-254.ec2.internal" podStartSLOduration=4.056101173 podStartE2EDuration="4.056101173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:17.041350956 +0000 UTC m=+4.545036821" watchObservedRunningTime="2026-07-17 01:13:17.056101173 +0000 UTC m=+4.559787036"
Jul 17 01:13:17.984470 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:17.984431    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:17.984962 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:17.984595    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:18.060874 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:18.060412    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal" event={"ID":"3060cc1582af17787ea2d991c3e8b345","Type":"ContainerStarted","Data":"d64d4a0d070c4901dacd7ecf20e593563662c46932c99a0caf3b8aeb0cfb921e"}
Jul 17 01:13:18.560016 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:18.559973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:18.560291 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:18.560054    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:18.560291 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.560180    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:18.560291 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.560240    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs podName:d4845309-a57a-4d8d-8b9e-6a23aeaab9c8 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:22.560221052 +0000 UTC m=+10.063906897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs") pod "network-metrics-daemon-nbf9g" (UID: "d4845309-a57a-4d8d-8b9e-6a23aeaab9c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:18.560707 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.560687    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 01:13:18.560806 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.560717    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 01:13:18.560806 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.560737    2582 projected.go:196] Error preparing data for projected volume kube-api-access-jws6f for pod openshift-network-diagnostics/network-check-target-sshtq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:18.560806 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.560784    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f podName:625502d2-d5e4-4d07-b8ec-830465847680 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:22.560768965 +0000 UTC m=+10.064454807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jws6f" (UniqueName: "kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f") pod "network-check-target-sshtq" (UID: "625502d2-d5e4-4d07-b8ec-830465847680") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:18.661227 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:18.660581    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:18.661227 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.660794    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:18.661227 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.660863    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret podName:b09b456f-062e-4b27-b5cc-4cfaff662554 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:22.660842626 +0000 UTC m=+10.164528568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret") pod "global-pull-secret-syncer-flbd5" (UID: "b09b456f-062e-4b27-b5cc-4cfaff662554") : object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:18.988477 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:18.987715    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:18.988477 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.987846    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:18.988477 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:18.988249    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:18.988477 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:18.988345    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:19.984632 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:19.984600    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:19.984824 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:19.984729    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:20.988059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:20.987347    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:20.988059 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:20.987482    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:20.988059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:20.987871    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:20.988059 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:20.987965    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:21.985068 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:21.984972    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:21.985231 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:21.985128    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:22.596208 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:22.596160    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:22.596678 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:22.596231    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:22.596678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.596338    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:22.596678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.596363    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 01:13:22.596678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.596389    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 01:13:22.596678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.596403    2582 projected.go:196] Error preparing data for projected volume kube-api-access-jws6f for pod openshift-network-diagnostics/network-check-target-sshtq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:22.596678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.596404    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs podName:d4845309-a57a-4d8d-8b9e-6a23aeaab9c8 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:30.596389399 +0000 UTC m=+18.100075238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs") pod "network-metrics-daemon-nbf9g" (UID: "d4845309-a57a-4d8d-8b9e-6a23aeaab9c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:22.596678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.596467    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f podName:625502d2-d5e4-4d07-b8ec-830465847680 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:30.596449793 +0000 UTC m=+18.100135641 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jws6f" (UniqueName: "kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f") pod "network-check-target-sshtq" (UID: "625502d2-d5e4-4d07-b8ec-830465847680") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:22.697462 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:22.697419    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:22.697667 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.697601    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:22.697727 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.697698    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret podName:b09b456f-062e-4b27-b5cc-4cfaff662554 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:30.69767791 +0000 UTC m=+18.201363756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret") pod "global-pull-secret-syncer-flbd5" (UID: "b09b456f-062e-4b27-b5cc-4cfaff662554") : object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:22.988557 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:22.988473    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:22.988711 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.988600    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:22.988997 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:22.988975    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:22.989113 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:22.989071    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:23.984966 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:23.984927    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:23.985454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:23.985060    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:24.984669 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:24.984636    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:24.984851 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:24.984766    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:24.985076 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:24.984637    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:24.985391 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:24.985148    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:25.985219 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:25.985176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:25.985732 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:25.985334    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:26.984424 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:26.984349    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:26.984616 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:26.984485    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:26.984616 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:26.984555    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:26.984729 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:26.984667    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:27.984442 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:27.984205    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:27.984912 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:27.984582    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:28.085695 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.085649    2582 generic.go:356] "Generic (PLEG): container finished" podID="c75f5bce-b93f-4a73-8119-63546da6f555" containerID="c105b806e9ddda1c5e2657d21f98db0d519c90330f66802d3b66de1db645dadf" exitCode=0
Jul 17 01:13:28.085847 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.085742    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" event={"ID":"c75f5bce-b93f-4a73-8119-63546da6f555","Type":"ContainerDied","Data":"c105b806e9ddda1c5e2657d21f98db0d519c90330f66802d3b66de1db645dadf"}
Jul 17 01:13:28.092457 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.092425    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jlch5" event={"ID":"a5f2c00d-8aa2-4181-99df-f099066c8a5b","Type":"ContainerStarted","Data":"30bdaeaf96f565c75849dfa7df3e52609bc80e7edf83a95b11db21e913e5bf05"}
Jul 17 01:13:28.094738 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.094710    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9xvcr" event={"ID":"8fe448c4-0355-4d02-8e50-e82e0a9c7ac5","Type":"ContainerStarted","Data":"fc68bf5cc30691170a3acb2e70dfef00263216b9920efe8c3f6fec3a8c2c402f"}
Jul 17 01:13:28.096621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.096590    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8g97m" event={"ID":"e600bd66-c18f-4b53-8f5d-dfff57448245","Type":"ContainerStarted","Data":"576867a71e0c2862487138b511c6295a0a6ebac70603244b58d634426b9bb385"}
Jul 17 01:13:28.098295 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.098273    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r" event={"ID":"08257645-b30f-4cb5-836c-f89f162cf8c8","Type":"ContainerStarted","Data":"75e4a965bba9f9fd14cbf46c8e050448bc7ec5f36a89e618037a8f5c1815ac41"}
Jul 17 01:13:28.099852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.099829    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5bmsd" event={"ID":"ff931d44-0f32-4884-87a5-0d657a1afda9","Type":"ContainerStarted","Data":"f58e7fb047f9522315bf43abcf33c2d8227ff375804c99bec053010da614b387"}
Jul 17 01:13:28.108662 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.108620    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-254.ec2.internal" podStartSLOduration=15.108600882 podStartE2EDuration="15.108600882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:18.076025708 +0000 UTC m=+5.579711578" watchObservedRunningTime="2026-07-17 01:13:28.108600882 +0000 UTC m=+15.612286745"
Jul 17 01:13:28.121973 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.121916    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9xvcr" podStartSLOduration=4.131445833 podStartE2EDuration="15.121898678s" totalImagesPullingTime="10.990452845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:28.121564912 +0000 UTC m=+15.625250777" watchObservedRunningTime="2026-07-17 01:13:28.121898678 +0000 UTC m=+15.625584544"
Jul 17 01:13:28.134527 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.134466    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jlch5" podStartSLOduration=4.389819467 podStartE2EDuration="15.134451049s" totalImagesPullingTime="10.744631582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:28.134022587 +0000 UTC m=+15.637708444" watchObservedRunningTime="2026-07-17 01:13:28.134451049 +0000 UTC m=+15.638136910"
Jul 17 01:13:28.155567 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.155480    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5bmsd" podStartSLOduration=4.403505558 podStartE2EDuration="15.155460687s" totalImagesPullingTime="10.751955129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:28.155294082 +0000 UTC m=+15.658979945" watchObservedRunningTime="2026-07-17 01:13:28.155460687 +0000 UTC m=+15.659146550"
Jul 17 01:13:28.984882 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.984848    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:28.984882 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:28.984879    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:28.985469 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:28.984980    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:28.985469 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:28.985069    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:29.103697 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:29.103565    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j5p9j" event={"ID":"e87c3885-4d60-419d-85f1-da2a84ae090a","Type":"ContainerStarted","Data":"0e1d8c682ac0923130da7062d8a228dbcf577950debb6e54a936bd21835b0731"}
Jul 17 01:13:29.117447 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:29.117387    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j5p9j" podStartSLOduration=5.370912604 podStartE2EDuration="16.117365174s" totalImagesPullingTime="10.74645257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:29.117121269 +0000 UTC m=+16.620807131" watchObservedRunningTime="2026-07-17 01:13:29.117365174 +0000 UTC m=+16.621051039"
Jul 17 01:13:29.118360 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:29.118063    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8g97m" podStartSLOduration=5.374109188 podStartE2EDuration="16.118048544s" totalImagesPullingTime="10.743939356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:28.169360839 +0000 UTC m=+15.673046701" watchObservedRunningTime="2026-07-17 01:13:29.118048544 +0000 UTC m=+16.621734405"
Jul 17 01:13:29.985043 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:29.985015    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:29.985579 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:29.985115    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:30.475612 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:30.475521    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:30.476215 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:30.476186    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:30.661295 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:30.661255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:30.661485 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:30.661331    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:30.661485 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.661436    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:30.661485 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.661442    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 01:13:30.661485 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.661470    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 01:13:30.661485 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.661502    2582 projected.go:196] Error preparing data for projected volume kube-api-access-jws6f for pod openshift-network-diagnostics/network-check-target-sshtq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:30.661717 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.661523    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs podName:d4845309-a57a-4d8d-8b9e-6a23aeaab9c8 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:46.66150719 +0000 UTC m=+34.165193046 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs") pod "network-metrics-daemon-nbf9g" (UID: "d4845309-a57a-4d8d-8b9e-6a23aeaab9c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:30.661717 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.661550    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f podName:625502d2-d5e4-4d07-b8ec-830465847680 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:46.661537776 +0000 UTC m=+34.165223632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jws6f" (UniqueName: "kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f") pod "network-check-target-sshtq" (UID: "625502d2-d5e4-4d07-b8ec-830465847680") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:30.761819 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:30.761718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:30.761987 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.761893    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:30.761987 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.761973    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret podName:b09b456f-062e-4b27-b5cc-4cfaff662554 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:46.761953821 +0000 UTC m=+34.265639675 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret") pod "global-pull-secret-syncer-flbd5" (UID: "b09b456f-062e-4b27-b5cc-4cfaff662554") : object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:30.984564 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:30.984527    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:30.984726 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:30.984578    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:30.984726 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.984667    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:30.984835 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:30.984794    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:31.108260 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:31.108172    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:31.108851 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:31.108753    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jlch5"
Jul 17 01:13:31.985198 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:31.985161    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:31.985361 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:31.985278    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:32.986216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:32.986184    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:32.986825 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:32.986307    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:32.986825 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:32.986363    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:32.986825 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:32.986504    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:33.984717 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:33.984666    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:33.984918 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:33.984832    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:34.984630 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:34.984598    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:34.985419 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:34.984711    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:34.985419 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:34.984598    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:34.985419 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:34.985071    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:35.985249 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:35.985215    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:35.985752 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:35.985341    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:36.984828 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:36.984788    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:36.984981 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:36.984931    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:36.985027 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:36.984973    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:36.985081 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:36.985064    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:37.202475 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:37.202324    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 01:13:37.985400 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:37.985159    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:37.985591 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:37.985544    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:37.985953 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:37.985850    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T01:13:37.202471038Z","UUID":"6cb60885-07c4-4351-8ff0-223a30be4dab","Handler":null,"Name":"","Endpoint":""}
Jul 17 01:13:37.987475 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:37.987452    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 01:13:37.987610 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:37.987526    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 01:13:38.124533 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.124478    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-blhf9" event={"ID":"db00c4f8-4c3f-4788-85a3-8fbd47cc557d","Type":"ContainerStarted","Data":"2ddb62c7f6fbf389d2b28a79cdbb26552c78f67538b32bd11123b24ac48895a4"}
Jul 17 01:13:38.127525 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.127480    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:13:38.127937 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.127898    2582 generic.go:356] "Generic (PLEG): container finished" podID="c8690b31-f48e-4eec-aafb-c91fb3cf442a" containerID="3e94d82bdbaf0e42463750b35adae4b0afb2ad63a762fc82ede836df4d47be62" exitCode=1
Jul 17 01:13:38.127937 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.127930    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"b852b54f0cdf4e5a6dbe29a25c69ded7d7cb006f0def3d32444bd94e470e5863"}
Jul 17 01:13:38.128108 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.128015    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"271335857ae92161c67ce7be61f5a5ea19fb91dc4898014d8bd2339b94b3e591"}
Jul 17 01:13:38.128108 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.128052    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"007437e5eaff9cf0e1c7208cf5c3e25926516b2454ded15cad480b108259a10c"}
Jul 17 01:13:38.128108 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.128087    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerDied","Data":"3e94d82bdbaf0e42463750b35adae4b0afb2ad63a762fc82ede836df4d47be62"}
Jul 17 01:13:38.128252 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.128118    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"39cabf1903187e08bdf78e768858af6f0a32e29e98b240d21c5479041e6fe526"}
Jul 17 01:13:38.130002 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.129982    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r" event={"ID":"08257645-b30f-4cb5-836c-f89f162cf8c8","Type":"ContainerStarted","Data":"2ebddc88a09b4995a88dc4b6d72805504fcae198b6ff5dd8d54c98abb3456f0e"}
Jul 17 01:13:38.140520 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.140465    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-blhf9" podStartSLOduration=4.261339213 podStartE2EDuration="25.140450662s" totalImagesPullingTime="20.879111449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:38.140190734 +0000 UTC m=+25.643876596" watchObservedRunningTime="2026-07-17 01:13:38.140450662 +0000 UTC m=+25.644136525"
Jul 17 01:13:38.985335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.985292    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:38.985840 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:38.985425    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:38.985840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:38.985475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:38.985840 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:38.985590    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:39.984915 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:39.984868    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:39.985106 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:39.985008    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:40.136450 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:40.136423    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:13:40.136961 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:40.136820    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"fa0f5771ddcbe97568e32296962335230e24ff9fda005ea5ace21e6eea56f0fa"}
Jul 17 01:13:40.138570 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:40.138544    2582 generic.go:356] "Generic (PLEG): container finished" podID="c75f5bce-b93f-4a73-8119-63546da6f555" containerID="c7baf2d8d905f17b0b817b2531877343f183a598cf4cbe45f4bded4d6399dc33" exitCode=0
Jul 17 01:13:40.138688 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:40.138576    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" event={"ID":"c75f5bce-b93f-4a73-8119-63546da6f555","Type":"ContainerDied","Data":"c7baf2d8d905f17b0b817b2531877343f183a598cf4cbe45f4bded4d6399dc33"}
Jul 17 01:13:40.984935 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:40.984893    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:40.985132 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:40.984893    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:40.985132 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:40.985028    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:40.985132 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:40.985069    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:41.144698 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:41.144651    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r" event={"ID":"08257645-b30f-4cb5-836c-f89f162cf8c8","Type":"ContainerStarted","Data":"3aeebdf56f1b3de7db11561bf8218df719d8332b825089b50c6797d907020766"}
Jul 17 01:13:41.168338 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:41.168289    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djm8r" podStartSLOduration=4.546859587 podStartE2EDuration="28.168273399s" totalImagesPullingTime="23.621413812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:41.1677363 +0000 UTC m=+28.671422162" watchObservedRunningTime="2026-07-17 01:13:41.168273399 +0000 UTC m=+28.671959261"
Jul 17 01:13:41.984936 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:41.984903    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:41.985125 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:41.985054    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:42.149297 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:42.149260    2582 generic.go:356] "Generic (PLEG): container finished" podID="c75f5bce-b93f-4a73-8119-63546da6f555" containerID="d8863ea050b2aff71f93d3006349ba241f0f1bdb5bc8d8f16f1d75c160096230" exitCode=0
Jul 17 01:13:42.149930 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:42.149339    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" event={"ID":"c75f5bce-b93f-4a73-8119-63546da6f555","Type":"ContainerDied","Data":"d8863ea050b2aff71f93d3006349ba241f0f1bdb5bc8d8f16f1d75c160096230"}
Jul 17 01:13:42.152719 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:42.152701    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:13:42.153120 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:42.153091    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"eaba999a9e4a167dd60a3f4ca1d9a6b264f4cc2b9aa17255aee5cccb59349430"}
Jul 17 01:13:42.985440 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:42.985400    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:42.985675 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:42.985400    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:42.985675 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:42.985534    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:42.985675 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:42.985574    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:43.984827 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:43.984633    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:43.985289 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:43.984907    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:44.160195 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.160160    2582 generic.go:356] "Generic (PLEG): container finished" podID="c75f5bce-b93f-4a73-8119-63546da6f555" containerID="040fa65a17aa208e8e4d38acf7538062c509b6b563d80a7681eb2347ac223927" exitCode=0
Jul 17 01:13:44.160367 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.160215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" event={"ID":"c75f5bce-b93f-4a73-8119-63546da6f555","Type":"ContainerDied","Data":"040fa65a17aa208e8e4d38acf7538062c509b6b563d80a7681eb2347ac223927"}
Jul 17 01:13:44.163993 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.163972    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:13:44.164261 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.164236    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"bd653cfe118a95534ccae3c307c8da06640fbedb8675e0a9bd2c3d1ea9ed4dc1"}
Jul 17 01:13:44.164477 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.164445    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:44.164583 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.164557    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:44.164752 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.164737    2582 scope.go:117] "RemoveContainer" containerID="3e94d82bdbaf0e42463750b35adae4b0afb2ad63a762fc82ede836df4d47be62"
Jul 17 01:13:44.181446 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.181428    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:44.984376 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.984341    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:44.984552 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:44.984384    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:44.984552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:44.984462    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:44.984653 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:44.984581    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:45.172912 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.171391    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:13:45.172912 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.171808    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" event={"ID":"c8690b31-f48e-4eec-aafb-c91fb3cf442a","Type":"ContainerStarted","Data":"f91fe68c24d29f5a6ac805ef8fc61ed269319a22fd7a3e661840422bdbb03373"}
Jul 17 01:13:45.172912 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.172038    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:45.190722 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.190696    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:13:45.200040 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.199981    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc" podStartSLOduration=11.270469635 podStartE2EDuration="32.199963814s" totalImagesPullingTime="20.929494179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:45.198577801 +0000 UTC m=+32.702263663" watchObservedRunningTime="2026-07-17 01:13:45.199963814 +0000 UTC m=+32.703649677"
Jul 17 01:13:45.586532 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.586484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nbf9g"]
Jul 17 01:13:45.586702 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.586621    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:45.586775 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:45.586720    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:45.589206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.589176    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-flbd5"]
Jul 17 01:13:45.589363 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.589296    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:45.589421 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:45.589369    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:45.598035 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.597791    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sshtq"]
Jul 17 01:13:45.598035 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:45.597902    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:45.598035 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:45.598000    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:46.685221 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:46.685188    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:46.685677 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:46.685268    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:46.685677 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.685329    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:46.685677 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.685391    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs podName:d4845309-a57a-4d8d-8b9e-6a23aeaab9c8 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:18.685376741 +0000 UTC m=+66.189062580 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs") pod "network-metrics-daemon-nbf9g" (UID: "d4845309-a57a-4d8d-8b9e-6a23aeaab9c8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 01:13:46.685677 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.685402    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 01:13:46.685677 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.685419    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 01:13:46.685677 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.685432    2582 projected.go:196] Error preparing data for projected volume kube-api-access-jws6f for pod openshift-network-diagnostics/network-check-target-sshtq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:46.685677 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.685483    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f podName:625502d2-d5e4-4d07-b8ec-830465847680 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:18.685467167 +0000 UTC m=+66.189153007 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jws6f" (UniqueName: "kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f") pod "network-check-target-sshtq" (UID: "625502d2-d5e4-4d07-b8ec-830465847680") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 01:13:46.786054 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:46.786017    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:46.786244 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.786138    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:46.786244 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.786204    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret podName:b09b456f-062e-4b27-b5cc-4cfaff662554 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:18.786186972 +0000 UTC m=+66.289872813 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret") pod "global-pull-secret-syncer-flbd5" (UID: "b09b456f-062e-4b27-b5cc-4cfaff662554") : object "kube-system"/"original-pull-secret" not registered
Jul 17 01:13:46.985006 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:46.984926    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:46.985006 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:46.984956    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:46.985176 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:46.984926    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:46.985176 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.985062    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:46.985241 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.985176    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:46.985278 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:46.985256    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:48.984601 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:48.984564    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:48.985067 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:48.984610    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:48.985067 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:48.984564    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:48.985067 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:48.984718    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nbf9g" podUID="d4845309-a57a-4d8d-8b9e-6a23aeaab9c8"
Jul 17 01:13:48.985067 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:48.984809    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sshtq" podUID="625502d2-d5e4-4d07-b8ec-830465847680"
Jul 17 01:13:48.985067 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:48.984892    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flbd5" podUID="b09b456f-062e-4b27-b5cc-4cfaff662554"
Jul 17 01:13:49.802054 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.801764    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-254.ec2.internal" event="NodeReady"
Jul 17 01:13:49.802198 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.802150    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 01:13:49.837183 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.836478    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"]
Jul 17 01:13:49.838886 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.838862    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"]
Jul 17 01:13:49.839037 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.839020    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:49.840371 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.840351    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7"]
Jul 17 01:13:49.840511 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.840476    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:49.842025 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.842007    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7"
Jul 17 01:13:49.842854 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.842832    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.842949 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.842855    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.842949 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.842876    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 17 01:13:49.842949 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.842938    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 17 01:13:49.843108 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.843018    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.843108 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.843030    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 17 01:13:49.843202 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.843144    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4sz7p\""
Jul 17 01:13:49.843303 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.843261    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.843359 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.843343    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 17 01:13:49.843359 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.843349    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5cxsx\""
Jul 17 01:13:49.844333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.844314    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-76xbz\""
Jul 17 01:13:49.844422 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.844377    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.845032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.845015    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.851958 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.851008    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"]
Jul 17 01:13:49.856839 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.856814    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-l9lkq"]
Jul 17 01:13:49.857964 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.857120    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:49.859517 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.859277    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"]
Jul 17 01:13:49.859632 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.859567    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:49.859714 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.859669    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.860363 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.860339    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 17 01:13:49.860517 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.860457    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fjhjw\""
Jul 17 01:13:49.860627 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.860531    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.862067 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.862042    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ghjrn"]
Jul 17 01:13:49.863441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.863417    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 17 01:13:49.865511 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.863635    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 17 01:13:49.867597 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.867576    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5f5b59d7fb-t6h6c"]
Jul 17 01:13:49.868356 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.868325    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2g6qb\""
Jul 17 01:13:49.868722 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.868703    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:49.870326 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.870307    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:49.875416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.875393    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.876045 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.876023    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.879699 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.878755    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78d7f6fd5-j9xgq"]
Jul 17 01:13:49.879699 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.878786    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 17 01:13:49.879699 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.879001    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qsv46\""
Jul 17 01:13:49.879699 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.879245    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:49.879699 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.879388    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-zn99v\""
Jul 17 01:13:49.879699 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.879533    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.881059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.881038    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.881148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.881076    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 17 01:13:49.881254 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.881239    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 17 01:13:49.881334 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.881315    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 17 01:13:49.881806 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.881579    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.881923 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.881842    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c"]
Jul 17 01:13:49.882188 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.882168    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.882383 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.882361    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:49.882550 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.882529    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 17 01:13:49.883645 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.883624    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.884579 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.884539    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tltqz\""
Jul 17 01:13:49.884686 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.884577    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 17 01:13:49.884686 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.884654    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.884873 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.884856    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 17 01:13:49.884931 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.884905    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 17 01:13:49.884979 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.884940    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 01:13:49.885033 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.884861    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 17 01:13:49.885320 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.885300    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 01:13:49.885399 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.885301    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65f8df49d6-jl8sm"]
Jul 17 01:13:49.885399 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.885367    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c"
Jul 17 01:13:49.886156 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.885692    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 01:13:49.887070 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.887051    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"]
Jul 17 01:13:49.887240 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.887225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:49.888593 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.888572    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 17 01:13:49.888974 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.888950    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.889070 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.888977    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5pmn2\""
Jul 17 01:13:49.889070 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.889054    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.889189 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.889169    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sjlm6\""
Jul 17 01:13:49.890360 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.889805    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7"]
Jul 17 01:13:49.890360 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.889842    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"]
Jul 17 01:13:49.890360 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.889863    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"]
Jul 17 01:13:49.890360 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.889885    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7m99b"]
Jul 17 01:13:49.890360 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.890118    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:49.901020 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.900991    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9mfwv\""
Jul 17 01:13:49.901292 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.901272    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 17 01:13:49.901416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.901375    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 17 01:13:49.902776 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.902202    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ks5wl"]
Jul 17 01:13:49.903374 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.902926    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 01:13:49.903374 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.902980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7m99b"
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904390    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-l9lkq"]
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904425    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"]
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904444    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"]
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904463    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ghjrn"]
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904508    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c"]
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904525    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7m99b"]
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904538    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"]
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904548    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78d7f6fd5-j9xgq"]
Jul 17 01:13:49.904571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904557    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:49.904963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904560    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65f8df49d6-jl8sm"]
Jul 17 01:13:49.904963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904804    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ks5wl"]
Jul 17 01:13:49.904963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.904831    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5f5b59d7fb-t6h6c"]
Jul 17 01:13:49.905428 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.905270    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 01:13:49.905428 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.905287    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 01:13:49.905428 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.905320    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 01:13:49.905428 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.905342    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-php8q\""
Jul 17 01:13:49.906743 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.906723    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 01:13:49.906835 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.906770    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lh9xl\""
Jul 17 01:13:49.907363 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:49.907343    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 01:13:50.006657 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006619    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b704339-44a2-4e1c-96b1-3950ca6a5d13-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006665    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50afa276-24c6-4cfe-b4e0-4d995de91eea-config\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006746    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006807    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-image-registry-private-configuration\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006879    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45zjc\" (UniqueName: \"kubernetes.io/projected/db8c951a-a427-4c1c-9afc-95d2388a8b6e-kube-api-access-45zjc\") pod \"network-check-source-7bc5b6bf85-5zw5c\" (UID: \"db8c951a-a427-4c1c-9afc-95d2388a8b6e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b704339-44a2-4e1c-96b1-3950ca6a5d13-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006949    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-installation-pull-secrets\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.006978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-serving-cert\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-trusted-ca\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-trusted-ca\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007055    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.007074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007075    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/438d40b6-c5d1-44e6-b796-c5f6ec836252-config\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007101    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7p8c\" (UniqueName: \"kubernetes.io/projected/59aea58b-5006-4136-a5e5-76b3cc757e1d-kube-api-access-d7p8c\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007128    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007143    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-registry-certificates\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007160    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-bound-sa-token\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007179    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/40a719e1-db20-4055-83a8-f522d628bf9a-ca-trust-extracted\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9ddv\" (UniqueName: \"kubernetes.io/projected/08b28450-98da-41fc-b59b-9ff001cb54c3-kube-api-access-k9ddv\") pod \"volume-data-source-validator-78cf4c6f6-p6wz7\" (UID: \"08b28450-98da-41fc-b59b-9ff001cb54c3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007331    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007400    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jm8v\" (UniqueName: \"kubernetes.io/projected/438d40b6-c5d1-44e6-b796-c5f6ec836252-kube-api-access-5jm8v\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-service-ca-bundle\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.007526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007469    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-certificates\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007593    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007633    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvlk4\" (UniqueName: \"kubernetes.io/projected/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-kube-api-access-cvlk4\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007660    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-snapshots\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007684    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-tmp-dir\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007787    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9pfq\" (UniqueName: \"kubernetes.io/projected/5b704339-44a2-4e1c-96b1-3950ca6a5d13-kube-api-access-l9pfq\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-default-certificate\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007842    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-installation-pull-secrets\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkfhv\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-kube-api-access-nkfhv\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.007893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007886    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-tmp\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007901    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6332dd34-b9db-4816-ac4b-04e7a4de05ca-cert\") pod \"ingress-canary-7m99b\" (UID: \"6332dd34-b9db-4816-ac4b-04e7a4de05ca\") " pod="openshift-ingress-canary/ingress-canary-7m99b"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-image-registry-private-configuration\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007944    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clrpc\" (UniqueName: \"kubernetes.io/projected/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-kube-api-access-clrpc\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.007979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50afa276-24c6-4cfe-b4e0-4d995de91eea-serving-cert\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008003    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76t4\" (UniqueName: \"kubernetes.io/projected/50afa276-24c6-4cfe-b4e0-4d995de91eea-kube-api-access-v76t4\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008022    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvvmh\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-kube-api-access-pvvmh\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-stats-auth\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/020ed824-8ef0-49a1-bf9f-9f184d9bd086-ca-trust-extracted\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-config-volume\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/438d40b6-c5d1-44e6-b796-c5f6ec836252-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2c77\" (UniqueName: \"kubernetes.io/projected/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-kube-api-access-q2c77\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.008306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50afa276-24c6-4cfe-b4e0-4d995de91eea-trusted-ca\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.008783 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008312    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-bound-sa-token\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.008783 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008358    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q68f\" (UniqueName: \"kubernetes.io/projected/6332dd34-b9db-4816-ac4b-04e7a4de05ca-kube-api-access-2q68f\") pod \"ingress-canary-7m99b\" (UID: \"6332dd34-b9db-4816-ac4b-04e7a4de05ca\") " pod="openshift-ingress-canary/ingress-canary-7m99b"
Jul 17 01:13:50.008783 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008375    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:50.008783 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008414    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4jfg\" (UniqueName: \"kubernetes.io/projected/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-kube-api-access-k4jfg\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.008783 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.008441    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.109598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.109506    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-trusted-ca\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.109598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.109553    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-trusted-ca\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.109598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.109582    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.109858 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.109611    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/438d40b6-c5d1-44e6-b796-c5f6ec836252-config\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.109858 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.109640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7p8c\" (UniqueName: \"kubernetes.io/projected/59aea58b-5006-4136-a5e5-76b3cc757e1d-kube-api-access-d7p8c\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.110278 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.110253    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.110343    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls podName:f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:50.610322841 +0000 UTC m=+38.114008685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls") pod "dns-default-ks5wl" (UID: "f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370") : secret "dns-default-metrics-tls" not found
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.109666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110445    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-registry-certificates\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.110643    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:13:50.61061183 +0000 UTC m=+38.114297686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : configmap references non-existent config key: service-ca.crt
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-bound-sa-token\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/40a719e1-db20-4055-83a8-f522d628bf9a-ca-trust-extracted\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110764    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110799    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9ddv\" (UniqueName: \"kubernetes.io/projected/08b28450-98da-41fc-b59b-9ff001cb54c3-kube-api-access-k9ddv\") pod \"volume-data-source-validator-78cf4c6f6-p6wz7\" (UID: \"08b28450-98da-41fc-b59b-9ff001cb54c3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110835    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110863    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-trusted-ca\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110874    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jm8v\" (UniqueName: \"kubernetes.io/projected/438d40b6-c5d1-44e6-b796-c5f6ec836252-kube-api-access-5jm8v\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-service-ca-bundle\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.110937    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-certificates\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.111216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.111039    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.111080    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvlk4\" (UniqueName: \"kubernetes.io/projected/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-kube-api-access-cvlk4\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.111111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-snapshots\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.111140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-tmp-dir\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.111335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.111366    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-registry-certificates\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.111409    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.111585    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert podName:e6dd53d6-002a-42bf-83bb-81cecb4c2e68 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:50.611562658 +0000 UTC m=+38.115248514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-7pr76" (UID: "e6dd53d6-002a-42bf-83bb-81cecb4c2e68") : secret "networking-console-plugin-cert" not found
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.111625    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.111671    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:13:50.611658587 +0000 UTC m=+38.115344428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : secret "router-metrics-certs-default" not found
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.111682    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 01:13:50.111772 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.111721    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls podName:be22baa4-1ff7-4d7f-86ac-6460e3a1d40e nodeName:}" failed. No retries permitted until 2026-07-17 01:13:50.611706435 +0000 UTC m=+38.115392279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jlqgm" (UID: "be22baa4-1ff7-4d7f-86ac-6460e3a1d40e") : secret "samples-operator-tls" not found
Jul 17 01:13:50.112212 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.111371    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9pfq\" (UniqueName: \"kubernetes.io/projected/5b704339-44a2-4e1c-96b1-3950ca6a5d13-kube-api-access-l9pfq\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.112212 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.111843    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-default-certificate\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.112212 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112017    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-tmp-dir\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.112212 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.112122    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:50.112212 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.112170    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls podName:7f62a271-55ea-42a7-8bcf-feaf6b7f80c2 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:50.612151092 +0000 UTC m=+38.115836934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ftjxh" (UID: "7f62a271-55ea-42a7-8bcf-feaf6b7f80c2") : secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:50.112212 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112208    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/40a719e1-db20-4055-83a8-f522d628bf9a-ca-trust-extracted\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.112483 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112327    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-trusted-ca\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.112483 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112422    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-service-ca-bundle\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.112483 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112475    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-snapshots\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.112630 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-installation-pull-secrets\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.112630 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkfhv\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-kube-api-access-nkfhv\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.112732 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-tmp\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.112732 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112659    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6332dd34-b9db-4816-ac4b-04e7a4de05ca-cert\") pod \"ingress-canary-7m99b\" (UID: \"6332dd34-b9db-4816-ac4b-04e7a4de05ca\") " pod="openshift-ingress-canary/ingress-canary-7m99b"
Jul 17 01:13:50.112732 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112705    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-image-registry-private-configuration\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.112872 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrpc\" (UniqueName: \"kubernetes.io/projected/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-kube-api-access-clrpc\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.112872 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50afa276-24c6-4cfe-b4e0-4d995de91eea-serving-cert\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.112872 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112815    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76t4\" (UniqueName: \"kubernetes.io/projected/50afa276-24c6-4cfe-b4e0-4d995de91eea-kube-api-access-v76t4\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.112872 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvvmh\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-kube-api-access-pvvmh\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.113059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112884    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.113059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112938    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-tmp\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.113059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-stats-auth\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.113059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/020ed824-8ef0-49a1-bf9f-9f184d9bd086-ca-trust-extracted\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.113059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113022    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-config-volume\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.113059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113056    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/438d40b6-c5d1-44e6-b796-c5f6ec836252-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.113270 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113093    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2c77\" (UniqueName: \"kubernetes.io/projected/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-kube-api-access-q2c77\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.113270 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113126    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50afa276-24c6-4cfe-b4e0-4d995de91eea-trusted-ca\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.113270 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-bound-sa-token\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.113270 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q68f\" (UniqueName: \"kubernetes.io/projected/6332dd34-b9db-4816-ac4b-04e7a4de05ca-kube-api-access-2q68f\") pod \"ingress-canary-7m99b\" (UID: \"6332dd34-b9db-4816-ac4b-04e7a4de05ca\") " pod="openshift-ingress-canary/ingress-canary-7m99b"
Jul 17 01:13:50.113270 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:50.113541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113295    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4jfg\" (UniqueName: \"kubernetes.io/projected/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-kube-api-access-k4jfg\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.113541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.113541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113363    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b704339-44a2-4e1c-96b1-3950ca6a5d13-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.113541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.113541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50afa276-24c6-4cfe-b4e0-4d995de91eea-config\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.113541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113454    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.113541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113484    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-image-registry-private-configuration\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.113802 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113555    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45zjc\" (UniqueName: \"kubernetes.io/projected/db8c951a-a427-4c1c-9afc-95d2388a8b6e-kube-api-access-45zjc\") pod \"network-check-source-7bc5b6bf85-5zw5c\" (UID: \"db8c951a-a427-4c1c-9afc-95d2388a8b6e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c"
Jul 17 01:13:50.113802 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b704339-44a2-4e1c-96b1-3950ca6a5d13-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.113802 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113614    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-installation-pull-secrets\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.113802 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.113641    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-serving-cert\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.114357 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.114088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/438d40b6-c5d1-44e6-b796-c5f6ec836252-config\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.114357 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.112706    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-certificates\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.115963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.115936    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/020ed824-8ef0-49a1-bf9f-9f184d9bd086-ca-trust-extracted\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.116464 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.116437    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-config-volume\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.117405 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.117387    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:50.117536 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.117524    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d7f6fd5-j9xgq: secret "image-registry-tls" not found
Jul 17 01:13:50.117686 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.117675    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls podName:020ed824-8ef0-49a1-bf9f-9f184d9bd086 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:50.617650975 +0000 UTC m=+38.121336834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls") pod "image-registry-78d7f6fd5-j9xgq" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086") : secret "image-registry-tls" not found
Jul 17 01:13:50.118552 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.118529    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.119342 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.119312    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/50afa276-24c6-4cfe-b4e0-4d995de91eea-config\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.119454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.119429    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/50afa276-24c6-4cfe-b4e0-4d995de91eea-serving-cert\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.119879    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b704339-44a2-4e1c-96b1-3950ca6a5d13-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.120006    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.120027    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f8df49d6-jl8sm: secret "image-registry-tls" not found
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.120087    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls podName:40a719e1-db20-4055-83a8-f522d628bf9a nodeName:}" failed. No retries permitted until 2026-07-17 01:13:50.620064711 +0000 UTC m=+38.123750550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls") pod "image-registry-65f8df49d6-jl8sm" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a") : secret "image-registry-tls" not found
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.120318    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-installation-pull-secrets\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.120516    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.120584    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-image-registry-private-configuration\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.120652    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6332dd34-b9db-4816-ac4b-04e7a4de05ca-cert\") pod \"ingress-canary-7m99b\" (UID: \"6332dd34-b9db-4816-ac4b-04e7a4de05ca\") " pod="openshift-ingress-canary/ingress-canary-7m99b"
Jul 17 01:13:50.121277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.121022    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-stats-auth\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.121948 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.121565    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9ddv\" (UniqueName: \"kubernetes.io/projected/08b28450-98da-41fc-b59b-9ff001cb54c3-kube-api-access-k9ddv\") pod \"volume-data-source-validator-78cf4c6f6-p6wz7\" (UID: \"08b28450-98da-41fc-b59b-9ff001cb54c3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7"
Jul 17 01:13:50.121948 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.121920    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:50.123475 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.123357    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50afa276-24c6-4cfe-b4e0-4d995de91eea-trusted-ca\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.123964 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.123882    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jm8v\" (UniqueName: \"kubernetes.io/projected/438d40b6-c5d1-44e6-b796-c5f6ec836252-kube-api-access-5jm8v\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.124920 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.124161    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/438d40b6-c5d1-44e6-b796-c5f6ec836252-serving-cert\") pod \"service-ca-operator-5bcbc94c78-4wn4j\" (UID: \"438d40b6-c5d1-44e6-b796-c5f6ec836252\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.124920 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.124228    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-serving-cert\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.125058 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.124953    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b704339-44a2-4e1c-96b1-3950ca6a5d13-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.126598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.125962    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvvmh\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-kube-api-access-pvvmh\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.126598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.126538    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvlk4\" (UniqueName: \"kubernetes.io/projected/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-kube-api-access-cvlk4\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:50.126757 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.126726    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-bound-sa-token\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.126848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.126820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-default-certificate\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.127871 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.127820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrpc\" (UniqueName: \"kubernetes.io/projected/32a81af9-c586-4ec0-b2a1-dca8d44df4fd-kube-api-access-clrpc\") pod \"insights-operator-555d9d7885-l9lkq\" (UID: \"32a81af9-c586-4ec0-b2a1-dca8d44df4fd\") " pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.130467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.130310    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76t4\" (UniqueName: \"kubernetes.io/projected/50afa276-24c6-4cfe-b4e0-4d995de91eea-kube-api-access-v76t4\") pod \"console-operator-565b7b569d-ghjrn\" (UID: \"50afa276-24c6-4cfe-b4e0-4d995de91eea\") " pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.130467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.130352    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-image-registry-private-configuration\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.130467 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.130385    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9pfq\" (UniqueName: \"kubernetes.io/projected/5b704339-44a2-4e1c-96b1-3950ca6a5d13-kube-api-access-l9pfq\") pod \"kube-storage-version-migrator-operator-7658d9b749-9xsvj\" (UID: \"5b704339-44a2-4e1c-96b1-3950ca6a5d13\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.131016 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.130952    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7p8c\" (UniqueName: \"kubernetes.io/projected/59aea58b-5006-4136-a5e5-76b3cc757e1d-kube-api-access-d7p8c\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.131262 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.131134    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkfhv\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-kube-api-access-nkfhv\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.131933 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.131910    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-bound-sa-token\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.132029 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.131947    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2c77\" (UniqueName: \"kubernetes.io/projected/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-kube-api-access-q2c77\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.132376 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.132358    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q68f\" (UniqueName: \"kubernetes.io/projected/6332dd34-b9db-4816-ac4b-04e7a4de05ca-kube-api-access-2q68f\") pod \"ingress-canary-7m99b\" (UID: \"6332dd34-b9db-4816-ac4b-04e7a4de05ca\") " pod="openshift-ingress-canary/ingress-canary-7m99b"
Jul 17 01:13:50.132462 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.132374    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4jfg\" (UniqueName: \"kubernetes.io/projected/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-kube-api-access-k4jfg\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.132861 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.132844    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45zjc\" (UniqueName: \"kubernetes.io/projected/db8c951a-a427-4c1c-9afc-95d2388a8b6e-kube-api-access-45zjc\") pod \"network-check-source-7bc5b6bf85-5zw5c\" (UID: \"db8c951a-a427-4c1c-9afc-95d2388a8b6e\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c"
Jul 17 01:13:50.133345 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.133324    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-installation-pull-secrets\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.167008 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.166980    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"
Jul 17 01:13:50.184626 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.184607    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7"
Jul 17 01:13:50.212360 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.212325    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-l9lkq"
Jul 17 01:13:50.220557 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.220530    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:13:50.226354 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.226334    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"
Jul 17 01:13:50.248342 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.248315    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c"
Jul 17 01:13:50.269185 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.269152    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7m99b"
Jul 17 01:13:50.516623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.516376    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j"]
Jul 17 01:13:50.519376 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.519226    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7m99b"]
Jul 17 01:13:50.522434 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:50.522315    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod438d40b6_c5d1_44e6_b796_c5f6ec836252.slice/crio-e5f7bb7876f2c16bae5a3ee238da3c2387619c0c884cc15e8792ca10d604e324 WatchSource:0}: Error finding container e5f7bb7876f2c16bae5a3ee238da3c2387619c0c884cc15e8792ca10d604e324: Status 404 returned error can't find the container with id e5f7bb7876f2c16bae5a3ee238da3c2387619c0c884cc15e8792ca10d604e324
Jul 17 01:13:50.526659 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:50.526628    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6332dd34_b9db_4816_ac4b_04e7a4de05ca.slice/crio-5f639a5ecabfffbf5e1ee6ed9728d257ebfbd5b79b439418b21a62105eff20a1 WatchSource:0}: Error finding container 5f639a5ecabfffbf5e1ee6ed9728d257ebfbd5b79b439418b21a62105eff20a1: Status 404 returned error can't find the container with id 5f639a5ecabfffbf5e1ee6ed9728d257ebfbd5b79b439418b21a62105eff20a1
Jul 17 01:13:50.530252 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.528578    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-ghjrn"]
Jul 17 01:13:50.531175 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.531150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj"]
Jul 17 01:13:50.534996 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:50.534970    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50afa276_24c6_4cfe_b4e0_4d995de91eea.slice/crio-6df1f3dba2ed0581f6428bcf20c67f526d7a7b2ce0f273ab1d347d3400237be9 WatchSource:0}: Error finding container 6df1f3dba2ed0581f6428bcf20c67f526d7a7b2ce0f273ab1d347d3400237be9: Status 404 returned error can't find the container with id 6df1f3dba2ed0581f6428bcf20c67f526d7a7b2ce0f273ab1d347d3400237be9
Jul 17 01:13:50.543890 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.543868    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7"]
Jul 17 01:13:50.543990 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.543923    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-l9lkq"]
Jul 17 01:13:50.552234 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.552209    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c"]
Jul 17 01:13:50.555149 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:50.555117    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a81af9_c586_4ec0_b2a1_dca8d44df4fd.slice/crio-fd29fffb868d1bd67f0f69f30795be253c3887a1622d5a1d284d7f610c5b73f0 WatchSource:0}: Error finding container fd29fffb868d1bd67f0f69f30795be253c3887a1622d5a1d284d7f610c5b73f0: Status 404 returned error can't find the container with id fd29fffb868d1bd67f0f69f30795be253c3887a1622d5a1d284d7f610c5b73f0
Jul 17 01:13:50.555793 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:50.555772    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb8c951a_a427_4c1c_9afc_95d2388a8b6e.slice/crio-b16c017bdcde25b67762139d66ba76326492c1b8172f0406b33cbc551b0f677b WatchSource:0}: Error finding container b16c017bdcde25b67762139d66ba76326492c1b8172f0406b33cbc551b0f677b: Status 404 returned error can't find the container with id b16c017bdcde25b67762139d66ba76326492c1b8172f0406b33cbc551b0f677b
Jul 17 01:13:50.557413 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:50.557306    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b28450_98da_41fc_b59b_9ff001cb54c3.slice/crio-00c6634fe1ec4c82a597b9340c4c4a8d3d369d31f968aa8836fda46ff960fcf3 WatchSource:0}: Error finding container 00c6634fe1ec4c82a597b9340c4c4a8d3d369d31f968aa8836fda46ff960fcf3: Status 404 returned error can't find the container with id 00c6634fe1ec4c82a597b9340c4c4a8d3d369d31f968aa8836fda46ff960fcf3
Jul 17 01:13:50.558449 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:13:50.557979    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b704339_44a2_4e1c_96b1_3950ca6a5d13.slice/crio-fdd795a642cc00505eea577af695d725d4427223c9f6edfb0f349e552b27e480 WatchSource:0}: Error finding container fdd795a642cc00505eea577af695d725d4427223c9f6edfb0f349e552b27e480: Status 404 returned error can't find the container with id fdd795a642cc00505eea577af695d725d4427223c9f6edfb0f349e552b27e480
Jul 17 01:13:50.620521 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.620438    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:50.620651 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.620515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:50.620651 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.620625    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:50.620651 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620631    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.620654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620683    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620710    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls podName:be22baa4-1ff7-4d7f-86ac-6460e3a1d40e nodeName:}" failed. No retries permitted until 2026-07-17 01:13:51.620689295 +0000 UTC m=+39.124375156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jlqgm" (UID: "be22baa4-1ff7-4d7f-86ac-6460e3a1d40e") : secret "samples-operator-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620741    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls podName:7f62a271-55ea-42a7-8bcf-feaf6b7f80c2 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:51.620727888 +0000 UTC m=+39.124413728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ftjxh" (UID: "7f62a271-55ea-42a7-8bcf-feaf6b7f80c2") : secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620741    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620759    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f8df49d6-jl8sm: secret "image-registry-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620759    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620776    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d7f6fd5-j9xgq: secret "image-registry-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620796    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls podName:40a719e1-db20-4055-83a8-f522d628bf9a nodeName:}" failed. No retries permitted until 2026-07-17 01:13:51.620784735 +0000 UTC m=+39.124470594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls") pod "image-registry-65f8df49d6-jl8sm" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a") : secret "image-registry-tls" not found
Jul 17 01:13:50.620799 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.620791    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620816    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls podName:020ed824-8ef0-49a1-bf9f-9f184d9bd086 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:51.620803115 +0000 UTC m=+39.124488954 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls") pod "image-registry-78d7f6fd5-j9xgq" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086") : secret "image-registry-tls" not found
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620839    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.620852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620865    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls podName:f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:51.620858441 +0000 UTC m=+39.124544281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls") pod "dns-default-ks5wl" (UID: "f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370") : secret "dns-default-metrics-tls" not found
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.620888    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620928    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:13:51.620916276 +0000 UTC m=+39.124602135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : configmap references non-existent config key: service-ca.crt
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.620940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.620992    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.621036    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert podName:e6dd53d6-002a-42bf-83bb-81cecb4c2e68 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:51.621025667 +0000 UTC m=+39.124711509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-7pr76" (UID: "e6dd53d6-002a-42bf-83bb-81cecb4c2e68") : secret "networking-console-plugin-cert" not found
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.621071    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 01:13:50.621185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:50.621130    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:13:51.621113278 +0000 UTC m=+39.124799141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : secret "router-metrics-certs-default" not found
Jul 17 01:13:50.984432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.984396    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:13:50.984599 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.984395    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:13:50.984599 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.984590    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:13:50.987318 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.987295    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 01:13:50.987466 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.987336    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 01:13:50.987531 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.987470    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hzllc\""
Jul 17 01:13:50.987564 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:50.987541    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dtqbv\""
Jul 17 01:13:51.186126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.186082    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" event={"ID":"50afa276-24c6-4cfe-b4e0-4d995de91eea","Type":"ContainerStarted","Data":"6df1f3dba2ed0581f6428bcf20c67f526d7a7b2ce0f273ab1d347d3400237be9"}
Jul 17 01:13:51.187380 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.187348    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj" event={"ID":"5b704339-44a2-4e1c-96b1-3950ca6a5d13","Type":"ContainerStarted","Data":"fdd795a642cc00505eea577af695d725d4427223c9f6edfb0f349e552b27e480"}
Jul 17 01:13:51.188630 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.188593    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-l9lkq" event={"ID":"32a81af9-c586-4ec0-b2a1-dca8d44df4fd","Type":"ContainerStarted","Data":"fd29fffb868d1bd67f0f69f30795be253c3887a1622d5a1d284d7f610c5b73f0"}
Jul 17 01:13:51.189894 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.189872    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7" event={"ID":"08b28450-98da-41fc-b59b-9ff001cb54c3","Type":"ContainerStarted","Data":"00c6634fe1ec4c82a597b9340c4c4a8d3d369d31f968aa8836fda46ff960fcf3"}
Jul 17 01:13:51.191143 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.191114    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j" event={"ID":"438d40b6-c5d1-44e6-b796-c5f6ec836252","Type":"ContainerStarted","Data":"e5f7bb7876f2c16bae5a3ee238da3c2387619c0c884cc15e8792ca10d604e324"}
Jul 17 01:13:51.192253 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.192194    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7m99b" event={"ID":"6332dd34-b9db-4816-ac4b-04e7a4de05ca","Type":"ContainerStarted","Data":"5f639a5ecabfffbf5e1ee6ed9728d257ebfbd5b79b439418b21a62105eff20a1"}
Jul 17 01:13:51.195273 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.195248    2582 generic.go:356] "Generic (PLEG): container finished" podID="c75f5bce-b93f-4a73-8119-63546da6f555" containerID="cc94966442f5ddb8357ab5026272effaced22cebc99fc69c74aab852ff34a907" exitCode=0
Jul 17 01:13:51.195352 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.195326    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" event={"ID":"c75f5bce-b93f-4a73-8119-63546da6f555","Type":"ContainerDied","Data":"cc94966442f5ddb8357ab5026272effaced22cebc99fc69c74aab852ff34a907"}
Jul 17 01:13:51.196560 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.196536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c" event={"ID":"db8c951a-a427-4c1c-9afc-95d2388a8b6e","Type":"ContainerStarted","Data":"b16c017bdcde25b67762139d66ba76326492c1b8172f0406b33cbc551b0f677b"}
Jul 17 01:13:51.631564 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.631348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:51.631564 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.631399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:51.631564 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.631438    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:51.631764 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.631607    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 01:13:51.631764 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.631687    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:13:53.631663819 +0000 UTC m=+41.135349676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : secret "router-metrics-certs-default" not found
Jul 17 01:13:51.631764 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.631688    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 01:13:51.631764 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.631734    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert podName:e6dd53d6-002a-42bf-83bb-81cecb4c2e68 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:53.631721536 +0000 UTC m=+41.135407375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-7pr76" (UID: "e6dd53d6-002a-42bf-83bb-81cecb4c2e68") : secret "networking-console-plugin-cert" not found
Jul 17 01:13:51.631918 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.631801    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 01:13:51.631918 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.631834    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls podName:be22baa4-1ff7-4d7f-86ac-6460e3a1d40e nodeName:}" failed. No retries permitted until 2026-07-17 01:13:53.631824361 +0000 UTC m=+41.135510220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jlqgm" (UID: "be22baa4-1ff7-4d7f-86ac-6460e3a1d40e") : secret "samples-operator-tls" not found
Jul 17 01:13:51.632008 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.631982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.631989    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.632110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632135    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls podName:7f62a271-55ea-42a7-8bcf-feaf6b7f80c2 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:53.632117855 +0000 UTC m=+41.135803712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ftjxh" (UID: "7f62a271-55ea-42a7-8bcf-feaf6b7f80c2") : secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.632178    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632199    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632213    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f8df49d6-jl8sm: secret "image-registry-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.632242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632261    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls podName:40a719e1-db20-4055-83a8-f522d628bf9a nodeName:}" failed. No retries permitted until 2026-07-17 01:13:53.63224607 +0000 UTC m=+41.135931915 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls") pod "image-registry-65f8df49d6-jl8sm" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a") : secret "image-registry-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:51.632294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632313    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632362    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls podName:f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:53.632332813 +0000 UTC m=+41.136018656 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls") pod "dns-default-ks5wl" (UID: "f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370") : secret "dns-default-metrics-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632419    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:13:53.632406125 +0000 UTC m=+41.136091982 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : configmap references non-existent config key: service-ca.crt
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632465    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632476    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d7f6fd5-j9xgq: secret "image-registry-tls" not found
Jul 17 01:13:51.632552 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:51.632528    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls podName:020ed824-8ef0-49a1-bf9f-9f184d9bd086 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:53.6325168 +0000 UTC m=+41.136202642 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls") pod "image-registry-78d7f6fd5-j9xgq" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086") : secret "image-registry-tls" not found
Jul 17 01:13:52.216348 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:52.215461    2582 generic.go:356] "Generic (PLEG): container finished" podID="c75f5bce-b93f-4a73-8119-63546da6f555" containerID="f013c10aee8830e4069b7b412474f8ecd5e94dfaed4970e8aef87c54604be076" exitCode=0
Jul 17 01:13:52.216348 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:52.215556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" event={"ID":"c75f5bce-b93f-4a73-8119-63546da6f555","Type":"ContainerDied","Data":"f013c10aee8830e4069b7b412474f8ecd5e94dfaed4970e8aef87c54604be076"}
Jul 17 01:13:52.956044 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:52.955986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:53.035228 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.035187    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:53.098441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.098377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:53.231464 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.231376    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" event={"ID":"c75f5bce-b93f-4a73-8119-63546da6f555","Type":"ContainerStarted","Data":"d62e6d7f1c11b09d1e762fa629f1b2353dd7ca23310c658e379e785fc7cf60ad"}
Jul 17 01:13:53.255106 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.255051    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pxbh2" podStartSLOduration=9.578237105 podStartE2EDuration="40.255032181s" totalImagesPullingTime="30.676795076s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:53.254515753 +0000 UTC m=+40.758201617" watchObservedRunningTime="2026-07-17 01:13:53.255032181 +0000 UTC m=+40.758718043"
Jul 17 01:13:53.377276 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.377209    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:53.477369 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.477329    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:53.655938 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.655862    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:53.655938 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.655914    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:53.656103 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.655947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:53.656103 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.655995    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:53.656103 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.656062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:53.656103 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.656084    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:53.656236 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.656128    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:53.656236 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:53.656148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:53.656298 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656267    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.656252198 +0000 UTC m=+45.159938040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : configmap references non-existent config key: service-ca.crt
Jul 17 01:13:53.656601 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656578    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:53.656717 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656628    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 01:13:53.656717 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656649    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls podName:7f62a271-55ea-42a7-8bcf-feaf6b7f80c2 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.6566289 +0000 UTC m=+45.160314754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ftjxh" (UID: "7f62a271-55ea-42a7-8bcf-feaf6b7f80c2") : secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:53.656717 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656689    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert podName:e6dd53d6-002a-42bf-83bb-81cecb4c2e68 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.656665151 +0000 UTC m=+45.160351007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-7pr76" (UID: "e6dd53d6-002a-42bf-83bb-81cecb4c2e68") : secret "networking-console-plugin-cert" not found
Jul 17 01:13:53.656717 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656690    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:53.656717 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656710    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f8df49d6-jl8sm: secret "image-registry-tls" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656723    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656742    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls podName:40a719e1-db20-4055-83a8-f522d628bf9a nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.656731825 +0000 UTC m=+45.160417685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls") pod "image-registry-65f8df49d6-jl8sm" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a") : secret "image-registry-tls" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656747    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656761    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls podName:be22baa4-1ff7-4d7f-86ac-6460e3a1d40e nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.656749256 +0000 UTC m=+45.160435115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jlqgm" (UID: "be22baa4-1ff7-4d7f-86ac-6460e3a1d40e") : secret "samples-operator-tls" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656761    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d7f6fd5-j9xgq: secret "image-registry-tls" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656581    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656795    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls podName:020ed824-8ef0-49a1-bf9f-9f184d9bd086 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.656785876 +0000 UTC m=+45.160471734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls") pod "image-registry-78d7f6fd5-j9xgq" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086") : secret "image-registry-tls" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656814    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.656800729 +0000 UTC m=+45.160486570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : secret "router-metrics-certs-default" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656821    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 01:13:53.656983 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:53.656853    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls podName:f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.656842275 +0000 UTC m=+45.160528116 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls") pod "dns-default-ks5wl" (UID: "f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370") : secret "dns-default-metrics-tls" not found
Jul 17 01:13:54.454933 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:54.454896    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:54.521327 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:54.521292    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:54.621939 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:54.621901    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:54.704657 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:54.704570    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:54.775597 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:54.775508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:55.518355 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:55.518309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:55.568246 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:55.568207    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:55.917192 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:55.917109    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:55.969437 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:55.969406    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:57.700383 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:57.700345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:13:57.700383 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:57.700383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:57.700409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:57.700429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:57.700453    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700518    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700528    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700544    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700577    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:14:05.700555518 +0000 UTC m=+53.204241375 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : configmap references non-existent config key: service-ca.crt
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700599    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls podName:f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:05.700591672 +0000 UTC m=+53.204277512 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls") pod "dns-default-ks5wl" (UID: "f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370") : secret "dns-default-metrics-tls" not found
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700609    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert podName:e6dd53d6-002a-42bf-83bb-81cecb4c2e68 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:05.700604134 +0000 UTC m=+53.204289974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-7pr76" (UID: "e6dd53d6-002a-42bf-83bb-81cecb4c2e68") : secret "networking-console-plugin-cert" not found
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700618    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls podName:be22baa4-1ff7-4d7f-86ac-6460e3a1d40e nodeName:}" failed. No retries permitted until 2026-07-17 01:14:05.700613372 +0000 UTC m=+53.204299212 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jlqgm" (UID: "be22baa4-1ff7-4d7f-86ac-6460e3a1d40e") : secret "samples-operator-tls" not found
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700619    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:57.700673    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700686    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:14:05.700659988 +0000 UTC m=+53.204345830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : secret "router-metrics-certs-default" not found
Jul 17 01:13:57.700951 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700723    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700763    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls podName:7f62a271-55ea-42a7-8bcf-feaf6b7f80c2 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:05.700752541 +0000 UTC m=+53.204438395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ftjxh" (UID: "7f62a271-55ea-42a7-8bcf-feaf6b7f80c2") : secret "cluster-monitoring-operator-tls" not found
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:57.700808    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:57.700837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700938    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700944    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700948    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d7f6fd5-j9xgq: secret "image-registry-tls" not found
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700958    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f8df49d6-jl8sm: secret "image-registry-tls" not found
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700986    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls podName:020ed824-8ef0-49a1-bf9f-9f184d9bd086 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:05.700979821 +0000 UTC m=+53.204665660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls") pod "image-registry-78d7f6fd5-j9xgq" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086") : secret "image-registry-tls" not found
Jul 17 01:13:57.701400 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:13:57.700997    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls podName:40a719e1-db20-4055-83a8-f522d628bf9a nodeName:}" failed. No retries permitted until 2026-07-17 01:14:05.700991679 +0000 UTC m=+53.204677519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls") pod "image-registry-65f8df49d6-jl8sm" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a") : secret "image-registry-tls" not found
Jul 17 01:13:59.254680 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.254372    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/0.log"
Jul 17 01:13:59.255597 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.254705    2582 generic.go:356] "Generic (PLEG): container finished" podID="50afa276-24c6-4cfe-b4e0-4d995de91eea" containerID="88ee56b1ef91cf79b172b6bea248a6d46179eb02d418ada974c64617d56705bc" exitCode=255
Jul 17 01:13:59.255597 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.254791    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" event={"ID":"50afa276-24c6-4cfe-b4e0-4d995de91eea","Type":"ContainerDied","Data":"88ee56b1ef91cf79b172b6bea248a6d46179eb02d418ada974c64617d56705bc"}
Jul 17 01:13:59.255597 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.255052    2582 scope.go:117] "RemoveContainer" containerID="88ee56b1ef91cf79b172b6bea248a6d46179eb02d418ada974c64617d56705bc"
Jul 17 01:13:59.257474 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.257443    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj" event={"ID":"5b704339-44a2-4e1c-96b1-3950ca6a5d13","Type":"ContainerStarted","Data":"cee53458eaadf8c796d4d23370914adf9fb3648d54a44154e53b5bb2b44da271"}
Jul 17 01:13:59.259791 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.259713    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-l9lkq" event={"ID":"32a81af9-c586-4ec0-b2a1-dca8d44df4fd","Type":"ContainerStarted","Data":"783d8c2f2ffbd8668882c96d27c160057ba3847a86cd5c2d39d8c040ba3c7cf7"}
Jul 17 01:13:59.261754 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.261727    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7" event={"ID":"08b28450-98da-41fc-b59b-9ff001cb54c3","Type":"ContainerStarted","Data":"8cb28e8c827d4ab6048666b036ccc2e96c569c3397b40546604e5f3561a39111"}
Jul 17 01:13:59.263824 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.263798    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j" event={"ID":"438d40b6-c5d1-44e6-b796-c5f6ec836252","Type":"ContainerStarted","Data":"9c47730dfcef26679b1dffd3538156a6dee1099c890368a8218ffc10b8c012eb"}
Jul 17 01:13:59.265928 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.265905    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7m99b" event={"ID":"6332dd34-b9db-4816-ac4b-04e7a4de05ca","Type":"ContainerStarted","Data":"ce502bce5ca21ab7001b8f2f31234cd1f6762cebf5f5594ef2c04704531b29f9"}
Jul 17 01:13:59.267596 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.267569    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c" event={"ID":"db8c951a-a427-4c1c-9afc-95d2388a8b6e","Type":"ContainerStarted","Data":"0081ab16d851900636933ae5c7d20b3c2b8c457e8c8d351ca4fac57c91defddc"}
Jul 17 01:13:59.305902 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.305850    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-p6wz7" podStartSLOduration=31.083545927 podStartE2EDuration="39.305830232s" totalImagesPullingTime="8.222284305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:59.287926835 +0000 UTC m=+46.791612697" watchObservedRunningTime="2026-07-17 01:13:59.305830232 +0000 UTC m=+46.809516095"
Jul 17 01:13:59.323728 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.323680    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j" podStartSLOduration=31.066897923 podStartE2EDuration="39.323657711s" totalImagesPullingTime="8.256759788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:59.307028459 +0000 UTC m=+46.810714322" watchObservedRunningTime="2026-07-17 01:13:59.323657711 +0000 UTC m=+46.827343554"
Jul 17 01:13:59.324727 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.324687    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-5zw5c" podStartSLOduration=31.099691349 podStartE2EDuration="39.324663182s" totalImagesPullingTime="8.224971833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:59.324080874 +0000 UTC m=+46.827766933" watchObservedRunningTime="2026-07-17 01:13:59.324663182 +0000 UTC m=+46.828349045"
Jul 17 01:13:59.339287 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.339245    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7m99b" podStartSLOduration=2.086278454 podStartE2EDuration="10.339225685s" totalImagesPullingTime="8.252947231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:59.338875981 +0000 UTC m=+46.842561844" watchObservedRunningTime="2026-07-17 01:13:59.339225685 +0000 UTC m=+46.842911548"
Jul 17 01:13:59.361012 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.360750    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-l9lkq" podStartSLOduration=31.13645163 podStartE2EDuration="39.360730267s" totalImagesPullingTime="8.224278637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:59.360297403 +0000 UTC m=+46.863983266" watchObservedRunningTime="2026-07-17 01:13:59.360730267 +0000 UTC m=+46.864416130"
Jul 17 01:13:59.379378 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.379327    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj" podStartSLOduration=31.157836047 podStartE2EDuration="39.379311948s" totalImagesPullingTime="8.221475901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:59.37854358 +0000 UTC m=+46.882229443" watchObservedRunningTime="2026-07-17 01:13:59.379311948 +0000 UTC m=+46.882997811"
Jul 17 01:13:59.396524 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.395903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:13:59.488850 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:13:59.488766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:00.221310 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.221275    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:14:00.221478 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.221339    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:14:00.231414 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.231379    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:00.272926 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.272894    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/1.log"
Jul 17 01:14:00.273422 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.273328    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/0.log"
Jul 17 01:14:00.273422 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.273359    2582 generic.go:356] "Generic (PLEG): container finished" podID="50afa276-24c6-4cfe-b4e0-4d995de91eea" containerID="6760e9ce73e093f99cc3139f790b4d960ba8f1664334d68112e042b59be8256e" exitCode=255
Jul 17 01:14:00.273544 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.273445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" event={"ID":"50afa276-24c6-4cfe-b4e0-4d995de91eea","Type":"ContainerDied","Data":"6760e9ce73e093f99cc3139f790b4d960ba8f1664334d68112e042b59be8256e"}
Jul 17 01:14:00.273544 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.273508    2582 scope.go:117] "RemoveContainer" containerID="88ee56b1ef91cf79b172b6bea248a6d46179eb02d418ada974c64617d56705bc"
Jul 17 01:14:00.273735 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.273716    2582 scope.go:117] "RemoveContainer" containerID="6760e9ce73e093f99cc3139f790b4d960ba8f1664334d68112e042b59be8256e"
Jul 17 01:14:00.273923 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:00.273900    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ghjrn_openshift-console-operator(50afa276-24c6-4cfe-b4e0-4d995de91eea)\"" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" podUID="50afa276-24c6-4cfe-b4e0-4d995de91eea"
Jul 17 01:14:00.367254 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.367207    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:00.762479 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.762439    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xtx4c"]
Jul 17 01:14:00.790522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.790440    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:00.793794 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.793769    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xtx4c"]
Jul 17 01:14:00.793924 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.793864    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.796134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.796113    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 01:14:00.796134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.796127    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 01:14:00.796290 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.796147    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h47bb\""
Jul 17 01:14:00.834870 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.834839    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6573de58-3849-4424-a40d-52deb60da3cb-data-volume\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.834998 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.834889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lldjf\" (UniqueName: \"kubernetes.io/projected/6573de58-3849-4424-a40d-52deb60da3cb-kube-api-access-lldjf\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.834998 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.834940    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6573de58-3849-4424-a40d-52deb60da3cb-crio-socket\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.834998 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.834976    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.835173 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.835021    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.935862 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.935814    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6573de58-3849-4424-a40d-52deb60da3cb-data-volume\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.936057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.935878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lldjf\" (UniqueName: \"kubernetes.io/projected/6573de58-3849-4424-a40d-52deb60da3cb-kube-api-access-lldjf\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.936057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.935918    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6573de58-3849-4424-a40d-52deb60da3cb-crio-socket\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.936057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.935968    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.936057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.936002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.936267 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:00.936140    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:00.936267 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.936145    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6573de58-3849-4424-a40d-52deb60da3cb-crio-socket\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.936267 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.936178    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6573de58-3849-4424-a40d-52deb60da3cb-data-volume\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:00.936267 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:00.936208    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:01.436192037 +0000 UTC m=+48.939877877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:00.936267 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:00.936145    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 01:14:00.936267 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:00.936244    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:01.436233701 +0000 UTC m=+48.939919577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : configmap "kube-rbac-proxy" not found
Jul 17 01:14:00.946344 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:00.946316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lldjf\" (UniqueName: \"kubernetes.io/projected/6573de58-3849-4424-a40d-52deb60da3cb-kube-api-access-lldjf\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:01.219384 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.219351    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj"]
Jul 17 01:14:01.237038 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.237009    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj"]
Jul 17 01:14:01.237200 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.237138    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj"
Jul 17 01:14:01.239434 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.239411    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 17 01:14:01.239598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.239484    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zhpz4\""
Jul 17 01:14:01.239598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.239515    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 17 01:14:01.278091 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.278066    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/1.log"
Jul 17 01:14:01.278419 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.278374    2582 scope.go:117] "RemoveContainer" containerID="6760e9ce73e093f99cc3139f790b4d960ba8f1664334d68112e042b59be8256e"
Jul 17 01:14:01.278572 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:01.278553    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-ghjrn_openshift-console-operator(50afa276-24c6-4cfe-b4e0-4d995de91eea)\"" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" podUID="50afa276-24c6-4cfe-b4e0-4d995de91eea"
Jul 17 01:14:01.340880 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.340844    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wbvl\" (UniqueName: \"kubernetes.io/projected/e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad-kube-api-access-9wbvl\") pod \"migrator-7cc9c7ffd7-rbqqj\" (UID: \"e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj"
Jul 17 01:14:01.345502 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.345460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:01.400027 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.399978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:01.442317 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.442282    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:01.442512 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.442324    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:01.442512 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.442378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wbvl\" (UniqueName: \"kubernetes.io/projected/e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad-kube-api-access-9wbvl\") pod \"migrator-7cc9c7ffd7-rbqqj\" (UID: \"e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj"
Jul 17 01:14:01.442512 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:01.442444    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:01.442512 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:01.442478    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 01:14:01.442663 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:01.442537    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:02.442517874 +0000 UTC m=+49.946203716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:01.442663 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:01.442559    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:02.4425451 +0000 UTC m=+49.946230965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : configmap "kube-rbac-proxy" not found
Jul 17 01:14:01.451180 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.451153    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wbvl\" (UniqueName: \"kubernetes.io/projected/e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad-kube-api-access-9wbvl\") pod \"migrator-7cc9c7ffd7-rbqqj\" (UID: \"e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj"
Jul 17 01:14:01.475554 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.475456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:01.547607 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.547572    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj"
Jul 17 01:14:01.683875 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.683847    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj"]
Jul 17 01:14:01.686090 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:01.686063    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8cc3b3c_99d5_482a_8ef9_c468ca5cb3ad.slice/crio-3871a049254fc70fbdb6ac0b897b9da1c609de55c9c458b8a01b9d9a205b4008 WatchSource:0}: Error finding container 3871a049254fc70fbdb6ac0b897b9da1c609de55c9c458b8a01b9d9a205b4008: Status 404 returned error can't find the container with id 3871a049254fc70fbdb6ac0b897b9da1c609de55c9c458b8a01b9d9a205b4008
Jul 17 01:14:01.908102 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.908029    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:01.962713 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.962687    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8g97m_e600bd66-c18f-4b53-8f5d-dfff57448245/dns-node-resolver/0.log"
Jul 17 01:14:01.977575 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:01.977542    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:02.182542 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:02.182511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:02.283297 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:02.283265    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj" event={"ID":"e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad","Type":"ContainerStarted","Data":"3871a049254fc70fbdb6ac0b897b9da1c609de55c9c458b8a01b9d9a205b4008"}
Jul 17 01:14:02.287817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:02.287789    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:02.452330 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:02.452249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:02.452469 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:02.452377    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 01:14:02.452469 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:02.452449    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:04.452431058 +0000 UTC m=+51.956116922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : configmap "kube-rbac-proxy" not found
Jul 17 01:14:02.452566 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:02.452538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:02.452639 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:02.452627    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:02.452689 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:02.452682    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:04.452656892 +0000 UTC m=+51.956342746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:02.562412 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:02.562383    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5bmsd_ff931d44-0f32-4884-87a5-0d657a1afda9/node-ca/0.log"
Jul 17 01:14:03.163423 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.163350    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7m99b_6332dd34-b9db-4816-ac4b-04e7a4de05ca/serve-healthcheck-canary/0.log"
Jul 17 01:14:03.784522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.784475    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fx8gh"]
Jul 17 01:14:03.802476 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.802452    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fx8gh"]
Jul 17 01:14:03.802644 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.802579    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.805046 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.805019    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 17 01:14:03.805200 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.805071    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 17 01:14:03.805905 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.805883    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 17 01:14:03.806049 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.805911    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 17 01:14:03.806049 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.805883    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xcszs\""
Jul 17 01:14:03.808437 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.808411    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:03.865372 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.865332    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:03.867945 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.867915    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/74cd9374-8966-4668-abaf-32aae5f5fdee-signing-key\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.868089 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.868027    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/74cd9374-8966-4668-abaf-32aae5f5fdee-signing-cabundle\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.868214 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.868197    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7bmn\" (UniqueName: \"kubernetes.io/projected/74cd9374-8966-4668-abaf-32aae5f5fdee-kube-api-access-h7bmn\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.968914 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.968878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7bmn\" (UniqueName: \"kubernetes.io/projected/74cd9374-8966-4668-abaf-32aae5f5fdee-kube-api-access-h7bmn\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.969053 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.968943    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/74cd9374-8966-4668-abaf-32aae5f5fdee-signing-key\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.969157 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.969131    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/74cd9374-8966-4668-abaf-32aae5f5fdee-signing-cabundle\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.969973 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.969950    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/74cd9374-8966-4668-abaf-32aae5f5fdee-signing-cabundle\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.971379 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.971359    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/74cd9374-8966-4668-abaf-32aae5f5fdee-signing-key\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:03.977179 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:03.977156    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7bmn\" (UniqueName: \"kubernetes.io/projected/74cd9374-8966-4668-abaf-32aae5f5fdee-kube-api-access-h7bmn\") pod \"service-ca-58598695f7-fx8gh\" (UID: \"74cd9374-8966-4668-abaf-32aae5f5fdee\") " pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:04.114080 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:04.113996    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-fx8gh"
Jul 17 01:14:04.422908 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:04.422879    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-fx8gh"]
Jul 17 01:14:04.425553 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:04.425527    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74cd9374_8966_4668_abaf_32aae5f5fdee.slice/crio-c9893c8647f64e5f64fb0d52004aa6b1726b2f6a7833ed1893edaf74954ec550 WatchSource:0}: Error finding container c9893c8647f64e5f64fb0d52004aa6b1726b2f6a7833ed1893edaf74954ec550: Status 404 returned error can't find the container with id c9893c8647f64e5f64fb0d52004aa6b1726b2f6a7833ed1893edaf74954ec550
Jul 17 01:14:04.474206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:04.474184    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:04.474279 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:04.474219    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:04.474352 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:04.474335    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:04.474398 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:04.474386    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 01:14:04.474398 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:04.474395    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:08.474379652 +0000 UTC m=+55.978065491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:04.474468 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:04.474428    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:08.474416982 +0000 UTC m=+55.978102854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : configmap "kube-rbac-proxy" not found
Jul 17 01:14:04.743777 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:04.743738    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:05.297897 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.297855    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj" event={"ID":"e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad","Type":"ContainerStarted","Data":"0a72aa441dabf841044ea7b7687dcf0ab189d9623ac22327b43f187c053ae769"}
Jul 17 01:14:05.298383 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.297951    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj" event={"ID":"e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad","Type":"ContainerStarted","Data":"be6b866f5fa7d55dcf07759fe10a055764f356965a4088c6f4e8a58883408804"}
Jul 17 01:14:05.299478 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.299449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-fx8gh" event={"ID":"74cd9374-8966-4668-abaf-32aae5f5fdee","Type":"ContainerStarted","Data":"4ec742c1f6a07af24ed94e25bb41037601dc0736a978e64b4903455ae729480f"}
Jul 17 01:14:05.299593 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.299513    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-fx8gh" event={"ID":"74cd9374-8966-4668-abaf-32aae5f5fdee","Type":"ContainerStarted","Data":"c9893c8647f64e5f64fb0d52004aa6b1726b2f6a7833ed1893edaf74954ec550"}
Jul 17 01:14:05.312761 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.312718    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-rbqqj" podStartSLOduration=1.699755329 podStartE2EDuration="4.312703925s" totalImagesPullingTime="2.612948596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:05.31225262 +0000 UTC m=+52.815938482" watchObservedRunningTime="2026-07-17 01:14:05.312703925 +0000 UTC m=+52.816389797"
Jul 17 01:14:05.326585 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.326542    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-fx8gh" podStartSLOduration=2.326532764 podStartE2EDuration="2.326532764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:05.32593307 +0000 UTC m=+52.829618933" watchObservedRunningTime="2026-07-17 01:14:05.326532764 +0000 UTC m=+52.830218625"
Jul 17 01:14:05.787175 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.787134    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:14:05.787175 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.787176    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787281    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787292    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78d7f6fd5-j9xgq: secret "image-registry-tls" not found
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787288    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787314    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f8df49d6-jl8sm: secret "image-registry-tls" not found
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.787309    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787347    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls podName:020ed824-8ef0-49a1-bf9f-9f184d9bd086 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:21.787333152 +0000 UTC m=+69.291018992 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls") pod "image-registry-78d7f6fd5-j9xgq" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086") : secret "image-registry-tls" not found
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.787366    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787397    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.787395    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787427    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls podName:40a719e1-db20-4055-83a8-f522d628bf9a nodeName:}" failed. No retries permitted until 2026-07-17 01:14:21.787406345 +0000 UTC m=+69.291092189 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls") pod "image-registry-65f8df49d6-jl8sm" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a") : secret "image-registry-tls" not found
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787448    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls podName:f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:21.787438921 +0000 UTC m=+69.291124762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls") pod "dns-default-ks5wl" (UID: "f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370") : secret "dns-default-metrics-tls" not found
Jul 17 01:14:05.787454 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787449    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787484    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:14:21.787467381 +0000 UTC m=+69.291153220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : configmap references non-existent config key: service-ca.crt
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.787540    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787571    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.787584    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787595    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs podName:59aea58b-5006-4136-a5e5-76b3cc757e1d nodeName:}" failed. No retries permitted until 2026-07-17 01:14:21.787587073 +0000 UTC m=+69.291272912 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs") pod "router-default-5f5b59d7fb-t6h6c" (UID: "59aea58b-5006-4136-a5e5-76b3cc757e1d") : secret "router-metrics-certs-default" not found
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787626    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert podName:e6dd53d6-002a-42bf-83bb-81cecb4c2e68 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:21.787606257 +0000 UTC m=+69.291292096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-7pr76" (UID: "e6dd53d6-002a-42bf-83bb-81cecb4c2e68") : secret "networking-console-plugin-cert" not found
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787650    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:05.787685    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787693    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls podName:be22baa4-1ff7-4d7f-86ac-6460e3a1d40e nodeName:}" failed. No retries permitted until 2026-07-17 01:14:21.787682955 +0000 UTC m=+69.291368795 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jlqgm" (UID: "be22baa4-1ff7-4d7f-86ac-6460e3a1d40e") : secret "samples-operator-tls" not found
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787731    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 17 01:14:05.788072 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:05.787772    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls podName:7f62a271-55ea-42a7-8bcf-feaf6b7f80c2 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:21.787761709 +0000 UTC m=+69.291447551 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-ftjxh" (UID: "7f62a271-55ea-42a7-8bcf-feaf6b7f80c2") : secret "cluster-monitoring-operator-tls" not found
Jul 17 01:14:06.351877 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:06.351842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:06.450148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:06.450116    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:06.550645 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:06.550614    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:06.654193 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:06.654099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:06.750719 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:06.750670    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:07.399148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:07.399112    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:07.500367 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:07.500330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:07.846079 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:07.846039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:07.947383 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:07.947351    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:08.004537 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:08.004507    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:08.243554 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:08.243521    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:08.514230 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:08.514144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:08.514230 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:08.514182    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:08.514678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:08.514271    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 01:14:08.514678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:08.514314    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:08.514678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:08.514326    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:16.514312595 +0000 UTC m=+64.017998435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : configmap "kube-rbac-proxy" not found
Jul 17 01:14:08.514678 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:08.514391    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:16.51437414 +0000 UTC m=+64.018060000 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:08.656999 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:08.656960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:09.044508 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:09.044463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:09.144593 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:09.144560    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:09.688120 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:09.688084    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:09.845309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:09.845278    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:10.047160 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.047074    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:10.161254 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.161223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:10.221514 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.221462    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:14:10.221684 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.221550    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:14:10.221963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.221946    2582 scope.go:117] "RemoveContainer" containerID="6760e9ce73e093f99cc3139f790b4d960ba8f1664334d68112e042b59be8256e"
Jul 17 01:14:10.316949 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.316870    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/1.log"
Jul 17 01:14:10.316949 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.316932    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" event={"ID":"50afa276-24c6-4cfe-b4e0-4d995de91eea","Type":"ContainerStarted","Data":"3347f22bfd8692bbc57add8170647fbbc25e6a4e8eb45497449b9e89cc242af3"}
Jul 17 01:14:10.317302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.317259    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:14:10.319134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.318801    2582 patch_prober.go:28] interesting pod/console-operator-565b7b569d-ghjrn container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body=
Jul 17 01:14:10.319134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.318855    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" podUID="50afa276-24c6-4cfe-b4e0-4d995de91eea" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused"
Jul 17 01:14:10.336796 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.336747    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" podStartSLOduration=42.08582358 podStartE2EDuration="50.336728575s" totalImagesPullingTime="8.250904995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:10.336029667 +0000 UTC m=+57.839715526" watchObservedRunningTime="2026-07-17 01:14:10.336728575 +0000 UTC m=+57.840414415"
Jul 17 01:14:10.453148 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.453114    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:10.647384 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:10.647295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:11.039733 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.039701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:11.099302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.099265    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:11.218790 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.218758    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:11.267795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.267763    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:11.322546 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.322471    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:14:11.322922 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.322909    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/1.log"
Jul 17 01:14:11.322982 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.322944    2582 generic.go:356] "Generic (PLEG): container finished" podID="50afa276-24c6-4cfe-b4e0-4d995de91eea" containerID="3347f22bfd8692bbc57add8170647fbbc25e6a4e8eb45497449b9e89cc242af3" exitCode=255
Jul 17 01:14:11.323026 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.323013    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" event={"ID":"50afa276-24c6-4cfe-b4e0-4d995de91eea","Type":"ContainerDied","Data":"3347f22bfd8692bbc57add8170647fbbc25e6a4e8eb45497449b9e89cc242af3"}
Jul 17 01:14:11.323082 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.323059    2582 scope.go:117] "RemoveContainer" containerID="6760e9ce73e093f99cc3139f790b4d960ba8f1664334d68112e042b59be8256e"
Jul 17 01:14:11.323260 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.323244    2582 scope.go:117] "RemoveContainer" containerID="3347f22bfd8692bbc57add8170647fbbc25e6a4e8eb45497449b9e89cc242af3"
Jul 17 01:14:11.323477 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:11.323448    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-ghjrn_openshift-console-operator(50afa276-24c6-4cfe-b4e0-4d995de91eea)\"" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" podUID="50afa276-24c6-4cfe-b4e0-4d995de91eea"
Jul 17 01:14:11.842577 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:11.842538    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:12.328165 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:12.328137    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:14:12.328574 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:12.328472    2582 scope.go:117] "RemoveContainer" containerID="3347f22bfd8692bbc57add8170647fbbc25e6a4e8eb45497449b9e89cc242af3"
Jul 17 01:14:12.328655 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:12.328637    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-ghjrn_openshift-console-operator(50afa276-24c6-4cfe-b4e0-4d995de91eea)\"" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" podUID="50afa276-24c6-4cfe-b4e0-4d995de91eea"
Jul 17 01:14:12.385855 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:12.385810    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:12.440783 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:12.440749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:12.543452 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:12.543421    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:12.641587 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:12.641507    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:12.693980 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:12.693937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:12.840786 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:12.840743    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:13.245157 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:13.245115    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:13.360462 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:13.360423    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:13.843279 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:13.843240    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:14.243327 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:14.243290    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:14.441876 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:14.441846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:14.840209 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:14.840172    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:15.644963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:15.644923    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:16.038790 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:16.038754    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:16.245327 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:16.245283    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:16.438220 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:16.438187    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:16.591735 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:16.591702    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:16.591735 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:16.591737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:16.591988 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:16.591879    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:16.591988 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:16.591955    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls podName:6573de58-3849-4424-a40d-52deb60da3cb nodeName:}" failed. No retries permitted until 2026-07-17 01:14:32.591934909 +0000 UTC m=+80.095620783 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xtx4c" (UID: "6573de58-3849-4424-a40d-52deb60da3cb") : secret "insights-runtime-extractor-tls" not found
Jul 17 01:14:16.592207 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:16.592187    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6573de58-3849-4424-a40d-52deb60da3cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:16.642911 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:16.642881    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:17.049932 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:17.049896    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:17.187506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:17.187467    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s6hjc"
Jul 17 01:14:18.711440 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.711386    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:14:18.711920 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.711469    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:14:18.713889 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.713868    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 01:14:18.714244 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.714222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jws6f\" (UniqueName: \"kubernetes.io/projected/625502d2-d5e4-4d07-b8ec-830465847680-kube-api-access-jws6f\") pod \"network-check-target-sshtq\" (UID: \"625502d2-d5e4-4d07-b8ec-830465847680\") " pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:14:18.724030 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.724001    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4845309-a57a-4d8d-8b9e-6a23aeaab9c8-metrics-certs\") pod \"network-metrics-daemon-nbf9g\" (UID: \"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8\") " pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:14:18.812234 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.812200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:14:18.814766 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.814746    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 01:14:18.824817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.824799    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b09b456f-062e-4b27-b5cc-4cfaff662554-original-pull-secret\") pod \"global-pull-secret-syncer-flbd5\" (UID: \"b09b456f-062e-4b27-b5cc-4cfaff662554\") " pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:14:18.895271 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.895235    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flbd5"
Jul 17 01:14:18.902890 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.902872    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dtqbv\""
Jul 17 01:14:18.907577 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.907556    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hzllc\""
Jul 17 01:14:18.911360 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.911312    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:14:18.916316 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:18.916295    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nbf9g"
Jul 17 01:14:19.042773 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.042742    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-flbd5"]
Jul 17 01:14:19.043361 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:19.043327    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09b456f_062e_4b27_b5cc_4cfaff662554.slice/crio-fdaeca9f21f0d327932f15421c696ef92ab95c208769afcdba6204001f044bce WatchSource:0}: Error finding container fdaeca9f21f0d327932f15421c696ef92ab95c208769afcdba6204001f044bce: Status 404 returned error can't find the container with id fdaeca9f21f0d327932f15421c696ef92ab95c208769afcdba6204001f044bce
Jul 17 01:14:19.058386 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.058358    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sshtq"]
Jul 17 01:14:19.061243 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:19.061216    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625502d2_d5e4_4d07_b8ec_830465847680.slice/crio-5efb9ca2fe192290ba227c6816fc97aa66d667946e4924c1fe0842879fabe468 WatchSource:0}: Error finding container 5efb9ca2fe192290ba227c6816fc97aa66d667946e4924c1fe0842879fabe468: Status 404 returned error can't find the container with id 5efb9ca2fe192290ba227c6816fc97aa66d667946e4924c1fe0842879fabe468
Jul 17 01:14:19.076926 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.076901    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nbf9g"]
Jul 17 01:14:19.080315 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:19.080293    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4845309_a57a_4d8d_8b9e_6a23aeaab9c8.slice/crio-c8e3cb081a01c4e291a8d9e577b18876cb59b9a7bf6d65ca66f51bd27651bcb0 WatchSource:0}: Error finding container c8e3cb081a01c4e291a8d9e577b18876cb59b9a7bf6d65ca66f51bd27651bcb0: Status 404 returned error can't find the container with id c8e3cb081a01c4e291a8d9e577b18876cb59b9a7bf6d65ca66f51bd27651bcb0
Jul 17 01:14:19.353390 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.353287    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbf9g" event={"ID":"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8","Type":"ContainerStarted","Data":"c8e3cb081a01c4e291a8d9e577b18876cb59b9a7bf6d65ca66f51bd27651bcb0"}
Jul 17 01:14:19.354763 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.354738    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sshtq" event={"ID":"625502d2-d5e4-4d07-b8ec-830465847680","Type":"ContainerStarted","Data":"b9b1458c03257482f2bf3bbd472353b0657609d2e2bf3691e059032afca92e04"}
Jul 17 01:14:19.354862 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.354787    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sshtq" event={"ID":"625502d2-d5e4-4d07-b8ec-830465847680","Type":"ContainerStarted","Data":"5efb9ca2fe192290ba227c6816fc97aa66d667946e4924c1fe0842879fabe468"}
Jul 17 01:14:19.354862 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.354851    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:14:19.356023 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.355993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-flbd5" event={"ID":"b09b456f-062e-4b27-b5cc-4cfaff662554","Type":"ContainerStarted","Data":"fdaeca9f21f0d327932f15421c696ef92ab95c208769afcdba6204001f044bce"}
Jul 17 01:14:19.370139 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:19.370100    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sshtq" podStartSLOduration=66.370088055 podStartE2EDuration="1m6.370088055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:19.368422604 +0000 UTC m=+66.872108483" watchObservedRunningTime="2026-07-17 01:14:19.370088055 +0000 UTC m=+66.873773894"
Jul 17 01:14:20.221749 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:20.221711    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:14:20.222183 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:20.222173    2582 scope.go:117] "RemoveContainer" containerID="3347f22bfd8692bbc57add8170647fbbc25e6a4e8eb45497449b9e89cc242af3"
Jul 17 01:14:20.222375 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:20.222356    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-ghjrn_openshift-console-operator(50afa276-24c6-4cfe-b4e0-4d995de91eea)\"" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" podUID="50afa276-24c6-4cfe-b4e0-4d995de91eea"
Jul 17 01:14:21.364922 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.364842    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbf9g" event={"ID":"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8","Type":"ContainerStarted","Data":"53a8167f4bdf575a7a3c175b56a9293314e8f6296a1051715f976c85f910faa6"}
Jul 17 01:14:21.364922 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.364896    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nbf9g" event={"ID":"d4845309-a57a-4d8d-8b9e-6a23aeaab9c8","Type":"ContainerStarted","Data":"7e5b42985cf00fca89463bc6e499b6c418793b890da10a0b07df7b6e6908c50b"}
Jul 17 01:14:21.381264 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.381218    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nbf9g" podStartSLOduration=66.738705017 podStartE2EDuration="1m8.381203688s" totalImagesPullingTime="1.642498671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:21.379445935 +0000 UTC m=+68.883131794" watchObservedRunningTime="2026-07-17 01:14:21.381203688 +0000 UTC m=+68.884889549"
Jul 17 01:14:21.839135 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.839097    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:14:21.839335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.839152    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:14:21.839335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.839204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:14:21.839335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.839235    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:21.839335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.839266    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:14:21.839335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.839301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:21.839632 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.839337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:14:21.839632 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.839387    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:14:21.840150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.840092    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59aea58b-5006-4136-a5e5-76b3cc757e1d-service-ca-bundle\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:21.842345 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.842316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370-metrics-tls\") pod \"dns-default-ks5wl\" (UID: \"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370\") " pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:14:21.842472 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.842393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"image-registry-78d7f6fd5-j9xgq\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") " pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:14:21.842945 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.842919    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f62a271-55ea-42a7-8bcf-feaf6b7f80c2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-ftjxh\" (UID: \"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:14:21.843034 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.842993    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"image-registry-65f8df49d6-jl8sm\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") " pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:14:21.843221 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.843173    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59aea58b-5006-4136-a5e5-76b3cc757e1d-metrics-certs\") pod \"router-default-5f5b59d7fb-t6h6c\" (UID: \"59aea58b-5006-4136-a5e5-76b3cc757e1d\") " pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:21.843651 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.843628    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/be22baa4-1ff7-4d7f-86ac-6460e3a1d40e-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jlqgm\" (UID: \"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:14:21.843841 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.843824    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e6dd53d6-002a-42bf-83bb-81cecb4c2e68-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-7pr76\" (UID: \"e6dd53d6-002a-42bf-83bb-81cecb4c2e68\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:14:21.956479 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.956442    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4sz7p\""
Jul 17 01:14:21.964733 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:21.964698    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"
Jul 17 01:14:22.006950 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.006919    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-fjhjw\""
Jul 17 01:14:22.015093 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.015062    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"
Jul 17 01:14:22.037304 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.037176    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-tltqz\""
Jul 17 01:14:22.046568 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.043611    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sjlm6\""
Jul 17 01:14:22.046568 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.044876    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:22.052082 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.051927    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:14:22.054401 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.054372    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:14:22.064144 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.063983    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9mfwv\""
Jul 17 01:14:22.072229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.071986    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"
Jul 17 01:14:22.077589 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.077344    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lh9xl\""
Jul 17 01:14:22.092287 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.090069    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:14:22.135549 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.135514    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh"]
Jul 17 01:14:22.270948 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.268058    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm"]
Jul 17 01:14:22.321954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.321444    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5f5b59d7fb-t6h6c"]
Jul 17 01:14:22.330061 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.329288    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78d7f6fd5-j9xgq"]
Jul 17 01:14:22.335469 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.335400    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76"]
Jul 17 01:14:22.337737 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:22.337702    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod020ed824_8ef0_49a1_bf9f_9f184d9bd086.slice/crio-a504269cb1ef5ff939951db25f2915c97943beb64d1e39cd94aa637ddf7d5065 WatchSource:0}: Error finding container a504269cb1ef5ff939951db25f2915c97943beb64d1e39cd94aa637ddf7d5065: Status 404 returned error can't find the container with id a504269cb1ef5ff939951db25f2915c97943beb64d1e39cd94aa637ddf7d5065
Jul 17 01:14:22.345054 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:22.344923    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6dd53d6_002a_42bf_83bb_81cecb4c2e68.slice/crio-1f99b7ee1eb7e6020e2f4e96d829908b8c528e24084cc416de1eeb9acc5dd032 WatchSource:0}: Error finding container 1f99b7ee1eb7e6020e2f4e96d829908b8c528e24084cc416de1eeb9acc5dd032: Status 404 returned error can't find the container with id 1f99b7ee1eb7e6020e2f4e96d829908b8c528e24084cc416de1eeb9acc5dd032
Jul 17 01:14:22.368002 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.367969    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65f8df49d6-jl8sm"]
Jul 17 01:14:22.374476 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.374442    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76" event={"ID":"e6dd53d6-002a-42bf-83bb-81cecb4c2e68","Type":"ContainerStarted","Data":"1f99b7ee1eb7e6020e2f4e96d829908b8c528e24084cc416de1eeb9acc5dd032"}
Jul 17 01:14:22.375897 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.375867    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq" event={"ID":"020ed824-8ef0-49a1-bf9f-9f184d9bd086","Type":"ContainerStarted","Data":"a504269cb1ef5ff939951db25f2915c97943beb64d1e39cd94aa637ddf7d5065"}
Jul 17 01:14:22.377093 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.377070    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm" event={"ID":"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e","Type":"ContainerStarted","Data":"e4b97b802ab5b30d9769e601f4f39ba97b8702c5120cfedb1fd36cf168a35d90"}
Jul 17 01:14:22.378086 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.378069    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh" event={"ID":"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2","Type":"ContainerStarted","Data":"e27eed0d8728ede13fac10d2833854e30b49e97e5a45112acc1bbb738afa6684"}
Jul 17 01:14:22.378927 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.378911    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c" event={"ID":"59aea58b-5006-4136-a5e5-76b3cc757e1d","Type":"ContainerStarted","Data":"5da1c16831a14fd83d3e6c096a984f789be0907350ef8450898c59233e065537"}
Jul 17 01:14:22.381901 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.381881    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ks5wl"]
Jul 17 01:14:22.924321 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.924232    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:22.976892 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:22.976855    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:23.737289 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:23.737253    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6bb9c9a_3b46_4a2f_9ec3_ee7fda9ab370.slice/crio-33b1a6a0573df9b08500a50488e0b14c2abac8d21745da4e8f5ae238ed31ee4f WatchSource:0}: Error finding container 33b1a6a0573df9b08500a50488e0b14c2abac8d21745da4e8f5ae238ed31ee4f: Status 404 returned error can't find the container with id 33b1a6a0573df9b08500a50488e0b14c2abac8d21745da4e8f5ae238ed31ee4f
Jul 17 01:14:24.246171 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.244957    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"]
Jul 17 01:14:24.247550 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.247513    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"]
Jul 17 01:14:24.247674 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.247635    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"
Jul 17 01:14:24.249884 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.249862    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"]
Jul 17 01:14:24.250045 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.250022    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.250130 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.250112    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 01:14:24.250410 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.250391    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2jxfl\""
Jul 17 01:14:24.254356 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.254334    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.254464 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.254395    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 01:14:24.256057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.255441    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 17 01:14:24.257522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.256710    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 01:14:24.257522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.257073    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 17 01:14:24.257522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.257370    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 17 01:14:24.257522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.257457    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 17 01:14:24.258388 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.258147    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 17 01:14:24.259272 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.258989    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 17 01:14:24.264164 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.263630    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"]
Jul 17 01:14:24.265722 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.265660    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"]
Jul 17 01:14:24.266431 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.266358    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"]
Jul 17 01:14:24.312206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.312126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:24.320157 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.320018    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78d7f6fd5-j9xgq"]
Jul 17 01:14:24.370506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.370022    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-ca\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.370506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.370085    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d512ce-306f-45a2-92a0-dd4191e32eea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.370506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.370116    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g68nk\" (UniqueName: \"kubernetes.io/projected/e4d512ce-306f-45a2-92a0-dd4191e32eea-kube-api-access-g68nk\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.370506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.370173    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g87wb\" (UniqueName: \"kubernetes.io/projected/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-kube-api-access-g87wb\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.370506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.370200    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff25b670-54b7-4ed5-a96c-2af1e097af3f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595cf9cf64-zkzc8\" (UID: \"ff25b670-54b7-4ed5-a96c-2af1e097af3f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"
Jul 17 01:14:24.370506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.370222    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-hub\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.370506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.370279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-tmp\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.370506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.370301    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.371559 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.371219    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbwkx\" (UniqueName: \"kubernetes.io/projected/ff25b670-54b7-4ed5-a96c-2af1e097af3f-kube-api-access-gbwkx\") pod \"managed-serviceaccount-addon-agent-595cf9cf64-zkzc8\" (UID: \"ff25b670-54b7-4ed5-a96c-2af1e097af3f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"
Jul 17 01:14:24.371559 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.371267    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.371559 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.371319    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.392753 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.392704    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm" event={"ID":"40a719e1-db20-4055-83a8-f522d628bf9a","Type":"ContainerStarted","Data":"c5923554da6d979c82367700ac9ed58eb0481ce1755052de64764c4593e02078"}
Jul 17 01:14:24.392926 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.392770    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm" event={"ID":"40a719e1-db20-4055-83a8-f522d628bf9a","Type":"ContainerStarted","Data":"0f8f9e222740e6430970b425bc388039d063597335892ae8ad8d89577ec71ac9"}
Jul 17 01:14:24.392926 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.392828    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:14:24.401000 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.400700    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq" event={"ID":"020ed824-8ef0-49a1-bf9f-9f184d9bd086","Type":"ContainerStarted","Data":"3e4f8e047a505ed4fa2340915cd8d16080e2d22d21c121c780881ed6dfe0b902"}
Jul 17 01:14:24.401186 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.401163    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:14:24.404084 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.403570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c" event={"ID":"59aea58b-5006-4136-a5e5-76b3cc757e1d","Type":"ContainerStarted","Data":"effb2f05ae0e109476344e7ce04b9726d991573b71feb02dccdf2447b422e1ec"}
Jul 17 01:14:24.406733 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.406706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:24.408310 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.408232    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ks5wl" event={"ID":"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370","Type":"ContainerStarted","Data":"33b1a6a0573df9b08500a50488e0b14c2abac8d21745da4e8f5ae238ed31ee4f"}
Jul 17 01:14:24.411224 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.410945    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-flbd5" event={"ID":"b09b456f-062e-4b27-b5cc-4cfaff662554","Type":"ContainerStarted","Data":"e65eddcd00fb773196a6b1ad79bcd03eb5d8411005296c0b0ddf33bbf08f472a"}
Jul 17 01:14:24.422222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.420287    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm" podStartSLOduration=63.420269438 podStartE2EDuration="1m3.420269438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:24.419883514 +0000 UTC m=+71.923569382" watchObservedRunningTime="2026-07-17 01:14:24.420269438 +0000 UTC m=+71.923955282"
Jul 17 01:14:24.445254 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.444933    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c" podStartSLOduration=64.444911316 podStartE2EDuration="1m4.444911316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:24.444418279 +0000 UTC m=+71.948104142" watchObservedRunningTime="2026-07-17 01:14:24.444911316 +0000 UTC m=+71.948597181"
Jul 17 01:14:24.462359 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.461287    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-flbd5" podStartSLOduration=66.725588079 podStartE2EDuration="1m11.461262236s" totalImagesPullingTime="4.735674157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:24.460275561 +0000 UTC m=+71.963961425" watchObservedRunningTime="2026-07-17 01:14:24.461262236 +0000 UTC m=+71.964948099"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g87wb\" (UniqueName: \"kubernetes.io/projected/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-kube-api-access-g87wb\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff25b670-54b7-4ed5-a96c-2af1e097af3f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595cf9cf64-zkzc8\" (UID: \"ff25b670-54b7-4ed5-a96c-2af1e097af3f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-hub\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472302    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-tmp\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472329    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472357    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwkx\" (UniqueName: \"kubernetes.io/projected/ff25b670-54b7-4ed5-a96c-2af1e097af3f-kube-api-access-gbwkx\") pod \"managed-serviceaccount-addon-agent-595cf9cf64-zkzc8\" (UID: \"ff25b670-54b7-4ed5-a96c-2af1e097af3f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472436    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472472    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-ca\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472557    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d512ce-306f-45a2-92a0-dd4191e32eea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.473041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.472589    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g68nk\" (UniqueName: \"kubernetes.io/projected/e4d512ce-306f-45a2-92a0-dd4191e32eea-kube-api-access-g68nk\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.477126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.474808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-tmp\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.477555 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.477512    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4d512ce-306f-45a2-92a0-dd4191e32eea-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.479122 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.479096    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-hub\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.479892 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.479857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-ca\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.481057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.481012    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.481955 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.481469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e4d512ce-306f-45a2-92a0-dd4191e32eea-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.481955 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.481910    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.483955 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.483893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff25b670-54b7-4ed5-a96c-2af1e097af3f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595cf9cf64-zkzc8\" (UID: \"ff25b670-54b7-4ed5-a96c-2af1e097af3f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"
Jul 17 01:14:24.500196 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.500129    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbwkx\" (UniqueName: \"kubernetes.io/projected/ff25b670-54b7-4ed5-a96c-2af1e097af3f-kube-api-access-gbwkx\") pod \"managed-serviceaccount-addon-agent-595cf9cf64-zkzc8\" (UID: \"ff25b670-54b7-4ed5-a96c-2af1e097af3f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"
Jul 17 01:14:24.500824 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.500779    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g68nk\" (UniqueName: \"kubernetes.io/projected/e4d512ce-306f-45a2-92a0-dd4191e32eea-kube-api-access-g68nk\") pod \"cluster-proxy-proxy-agent-cb446cb7f-z6wxj\" (UID: \"e4d512ce-306f-45a2-92a0-dd4191e32eea\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.511205 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.511152    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g87wb\" (UniqueName: \"kubernetes.io/projected/92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84-kube-api-access-g87wb\") pod \"klusterlet-addon-workmgr-6d6dc7bf-pnxkw\" (UID: \"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.531961 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.531839    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:24.580511 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.580466    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"
Jul 17 01:14:24.595059 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.594836    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"
Jul 17 01:14:24.613689 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.613452    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:24.617669 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.617457    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:24.714962 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:24.714905    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:25.048186 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.048071    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:25.055369 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.055340    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:25.085683 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.085625    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq" podStartSLOduration=65.085599071 podStartE2EDuration="1m5.085599071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:24.50519929 +0000 UTC m=+72.008885153" watchObservedRunningTime="2026-07-17 01:14:25.085599071 +0000 UTC m=+72.589284935"
Jul 17 01:14:25.361710 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.361682    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8"]
Jul 17 01:14:25.375443 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.375419    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"]
Jul 17 01:14:25.378567 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.378549    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj"]
Jul 17 01:14:25.419672 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.419632    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76" event={"ID":"e6dd53d6-002a-42bf-83bb-81cecb4c2e68","Type":"ContainerStarted","Data":"24b6009eb1b7b66492a9cffc6b21d8d61f4ea1316a0ff2d02dfbfccb8f7df83e"}
Jul 17 01:14:25.421052 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.421028    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:25.423134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.423116    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5f5b59d7fb-t6h6c"
Jul 17 01:14:25.436768 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.435038    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-7pr76" podStartSLOduration=56.781898462 podStartE2EDuration="59.435018743s" totalImagesPullingTime="2.653120281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:25.434550535 +0000 UTC m=+72.938236398" watchObservedRunningTime="2026-07-17 01:14:25.435018743 +0000 UTC m=+72.938704607"
Jul 17 01:14:25.526797 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.526768    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:25.631458 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.631378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:25.670301 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:25.670267    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4d512ce_306f_45a2_92a0_dd4191e32eea.slice/crio-a2f38051fd9457e6977a93359624f0dc700ec968fad75cb6d6e0c34a59a31d11 WatchSource:0}: Error finding container a2f38051fd9457e6977a93359624f0dc700ec968fad75cb6d6e0c34a59a31d11: Status 404 returned error can't find the container with id a2f38051fd9457e6977a93359624f0dc700ec968fad75cb6d6e0c34a59a31d11
Jul 17 01:14:25.730651 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:25.730568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:26.150309 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:26.150265    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff25b670_54b7_4ed5_a96c_2af1e097af3f.slice/crio-b0912ed52520873bba14a864dc3a06e4e4c73161dbf0a85fa5652b5b2a4187d1 WatchSource:0}: Error finding container b0912ed52520873bba14a864dc3a06e4e4c73161dbf0a85fa5652b5b2a4187d1: Status 404 returned error can't find the container with id b0912ed52520873bba14a864dc3a06e4e4c73161dbf0a85fa5652b5b2a4187d1
Jul 17 01:14:26.378639 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:26.378592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log" limit=1024
Jul 17 01:14:26.424954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:26.424861    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8" event={"ID":"ff25b670-54b7-4ed5-a96c-2af1e097af3f","Type":"ContainerStarted","Data":"b0912ed52520873bba14a864dc3a06e4e4c73161dbf0a85fa5652b5b2a4187d1"}
Jul 17 01:14:26.426354 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:26.426298    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj" event={"ID":"e4d512ce-306f-45a2-92a0-dd4191e32eea","Type":"ContainerStarted","Data":"a2f38051fd9457e6977a93359624f0dc700ec968fad75cb6d6e0c34a59a31d11"}
Jul 17 01:14:26.608683 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:26.608650    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92930a25_1ebc_40d5_bdd8_0eb1ff3f3d84.slice/crio-f413384d480cc183fe50c9ffee3a1a6af44302ce0977910d84603a3e91a7b090 WatchSource:0}: Error finding container f413384d480cc183fe50c9ffee3a1a6af44302ce0977910d84603a3e91a7b090: Status 404 returned error can't find the container with id f413384d480cc183fe50c9ffee3a1a6af44302ce0977910d84603a3e91a7b090
Jul 17 01:14:27.230820 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.230753    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"]
Jul 17 01:14:27.235981 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.235144    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"
Jul 17 01:14:27.238834 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.238176    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 17 01:14:27.238834 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.238479    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bwmk4\""
Jul 17 01:14:27.241760 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.241702    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"]
Jul 17 01:14:27.305752 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.305566    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4f66694f-050f-4a99-a9d4-a4f26f4a2550-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-cttwl\" (UID: \"4f66694f-050f-4a99-a9d4-a4f26f4a2550\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"
Jul 17 01:14:27.407085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.406633    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4f66694f-050f-4a99-a9d4-a4f26f4a2550-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-cttwl\" (UID: \"4f66694f-050f-4a99-a9d4-a4f26f4a2550\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"
Jul 17 01:14:27.407085 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:27.406796    2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 01:14:27.407085 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:14:27.406867    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f66694f-050f-4a99-a9d4-a4f26f4a2550-tls-certificates podName:4f66694f-050f-4a99-a9d4-a4f26f4a2550 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:27.90684835 +0000 UTC m=+75.410534206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/4f66694f-050f-4a99-a9d4-a4f26f4a2550-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-cttwl" (UID: "4f66694f-050f-4a99-a9d4-a4f26f4a2550") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 01:14:27.437024 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.436026    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ks5wl" event={"ID":"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370","Type":"ContainerStarted","Data":"aa32e080762a23e2e41b90f47450465fd1f1a4ab9c5391af134c5de78cbb4042"}
Jul 17 01:14:27.437024 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.436089    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ks5wl" event={"ID":"f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370","Type":"ContainerStarted","Data":"1dd2ed0b52c4346cc9cdb468a951ea0fcb7081888a76dc2a3db97961fa4437fa"}
Jul 17 01:14:27.437024 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.436959    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:14:27.441316 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.441282    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw" event={"ID":"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84","Type":"ContainerStarted","Data":"f413384d480cc183fe50c9ffee3a1a6af44302ce0977910d84603a3e91a7b090"}
Jul 17 01:14:27.445134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.445044    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm" event={"ID":"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e","Type":"ContainerStarted","Data":"f030653a8adc3838b7f0f3ce042e8427fa5b482d4fae6ee23fc1fd2ba07056b8"}
Jul 17 01:14:27.445134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.445096    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm" event={"ID":"be22baa4-1ff7-4d7f-86ac-6460e3a1d40e","Type":"ContainerStarted","Data":"15ceb5ce6dc80c5278b12bdc51f8e94807ef50e0c6382898dece2b884d40ee0f"}
Jul 17 01:14:27.453165 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.452844    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh" event={"ID":"7f62a271-55ea-42a7-8bcf-feaf6b7f80c2","Type":"ContainerStarted","Data":"ef2863be952200116f235b8b44114ef5d1ce2c05a950bd271489cbec058ace0d"}
Jul 17 01:14:27.454962 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.454909    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ks5wl" podStartSLOduration=35.511007451 podStartE2EDuration="38.45488758s" totalImagesPullingTime="2.943880129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:27.453588435 +0000 UTC m=+74.957274295" watchObservedRunningTime="2026-07-17 01:14:27.45488758 +0000 UTC m=+74.958573442"
Jul 17 01:14:27.471301 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.470875    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-ftjxh" podStartSLOduration=62.945139895 podStartE2EDuration="1m7.470854726s" totalImagesPullingTime="4.525714831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:27.468707609 +0000 UTC m=+74.972393472" watchObservedRunningTime="2026-07-17 01:14:27.470854726 +0000 UTC m=+74.974540590"
Jul 17 01:14:27.913229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.912579    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4f66694f-050f-4a99-a9d4-a4f26f4a2550-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-cttwl\" (UID: \"4f66694f-050f-4a99-a9d4-a4f26f4a2550\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"
Jul 17 01:14:27.922959 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:27.922895    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4f66694f-050f-4a99-a9d4-a4f26f4a2550-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-cttwl\" (UID: \"4f66694f-050f-4a99-a9d4-a4f26f4a2550\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"
Jul 17 01:14:28.155606 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:28.155569    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"
Jul 17 01:14:28.360860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:28.360232    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jlqgm" podStartSLOduration=64.027950382 podStartE2EDuration="1m8.360205176s" totalImagesPullingTime="4.332254794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:27.485065268 +0000 UTC m=+74.988751112" watchObservedRunningTime="2026-07-17 01:14:28.360205176 +0000 UTC m=+75.863891038"
Jul 17 01:14:28.361827 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:28.361666    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"]
Jul 17 01:14:28.364753 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:28.364716    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f66694f_050f_4a99_a9d4_a4f26f4a2550.slice/crio-80381f1faab6c5c2d65c60c67b81cbba3dfe35c4c68f69b3bd423077776673ab WatchSource:0}: Error finding container 80381f1faab6c5c2d65c60c67b81cbba3dfe35c4c68f69b3bd423077776673ab: Status 404 returned error can't find the container with id 80381f1faab6c5c2d65c60c67b81cbba3dfe35c4c68f69b3bd423077776673ab
Jul 17 01:14:28.469553 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:28.468599    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl" event={"ID":"4f66694f-050f-4a99-a9d4-a4f26f4a2550","Type":"ContainerStarted","Data":"80381f1faab6c5c2d65c60c67b81cbba3dfe35c4c68f69b3bd423077776673ab"}
Jul 17 01:14:32.664280 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:32.664239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:32.666638 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:32.666614    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6573de58-3849-4424-a40d-52deb60da3cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtx4c\" (UID: \"6573de58-3849-4424-a40d-52deb60da3cb\") " pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:32.906206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:32.906173    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h47bb\""
Jul 17 01:14:32.914463 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:32.914405    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xtx4c"
Jul 17 01:14:32.986506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:32.986465    2582 scope.go:117] "RemoveContainer" containerID="3347f22bfd8692bbc57add8170647fbbc25e6a4e8eb45497449b9e89cc242af3"
Jul 17 01:14:33.621987 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:33.621957    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xtx4c"]
Jul 17 01:14:33.678767 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:33.678656    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6573de58_3849_4424_a40d_52deb60da3cb.slice/crio-c8c0a359dbaa7e281dfab43fc42a5645ad76423546317809fcf831ae0bc6703f WatchSource:0}: Error finding container c8c0a359dbaa7e281dfab43fc42a5645ad76423546317809fcf831ae0bc6703f: Status 404 returned error can't find the container with id c8c0a359dbaa7e281dfab43fc42a5645ad76423546317809fcf831ae0bc6703f
Jul 17 01:14:34.496095 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.496056    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw" event={"ID":"92930a25-1ebc-40d5-bdd8-0eb1ff3f3d84","Type":"ContainerStarted","Data":"41f636fa0bb4050b445c6a4feb3fdc1eeb43f93366ba8d6eb64a0cbac8d0659b"}
Jul 17 01:14:34.496606 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.496396    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:34.498482 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.498460    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw"
Jul 17 01:14:34.498717 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.498694    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj" event={"ID":"e4d512ce-306f-45a2-92a0-dd4191e32eea","Type":"ContainerStarted","Data":"ea7d8030f447e5d2d978502a9b8232001f960d609e2b15cb6e4398491b442148"}
Jul 17 01:14:34.498793 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.498730    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj" event={"ID":"e4d512ce-306f-45a2-92a0-dd4191e32eea","Type":"ContainerStarted","Data":"8ea51fd30a654f5cf00a330823d6904668824f0aef389674576406df350e1d85"}
Jul 17 01:14:34.498793 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.498747    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj" event={"ID":"e4d512ce-306f-45a2-92a0-dd4191e32eea","Type":"ContainerStarted","Data":"8454928525dcdbd3d31248c92e1df64e135ef026f976ab89e904ea773ca6548a"}
Jul 17 01:14:34.501048 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.501021    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:14:34.501156 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.501114    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn" event={"ID":"50afa276-24c6-4cfe-b4e0-4d995de91eea","Type":"ContainerStarted","Data":"297a59051d2e07a91986f2f0e51f76bc80e7c402bf45e018e3c0410a7cd80640"}
Jul 17 01:14:34.501378 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.501359    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:14:34.504447 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.504418    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl" event={"ID":"4f66694f-050f-4a99-a9d4-a4f26f4a2550","Type":"ContainerStarted","Data":"537330b6a6a7bfd016c3ce2617aa086627a9e326d8f3b2b7d599e045893529e2"}
Jul 17 01:14:34.506051 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.506031    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"
Jul 17 01:14:34.507895 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.507876    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-ghjrn"
Jul 17 01:14:34.508423 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.508049    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8" event={"ID":"ff25b670-54b7-4ed5-a96c-2af1e097af3f","Type":"ContainerStarted","Data":"62de9413983b1f5c12083b7da86542c0b33fbdd3b2765c8c64ccd07c1ab13a2b"}
Jul 17 01:14:34.510436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.510387    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl"
Jul 17 01:14:34.510623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.510600    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xtx4c" event={"ID":"6573de58-3849-4424-a40d-52deb60da3cb","Type":"ContainerStarted","Data":"c8c0a359dbaa7e281dfab43fc42a5645ad76423546317809fcf831ae0bc6703f"}
Jul 17 01:14:34.534221 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.533595    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d6dc7bf-pnxkw" podStartSLOduration=3.737465576 podStartE2EDuration="10.533572688s" totalImagesPullingTime="6.796107112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:34.514429124 +0000 UTC m=+82.018114987" watchObservedRunningTime="2026-07-17 01:14:34.533572688 +0000 UTC m=+82.037258553"
Jul 17 01:14:34.549573 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.549317    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cb446cb7f-z6wxj" podStartSLOduration=2.751328804 podStartE2EDuration="10.54929401s" totalImagesPullingTime="7.797965206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:34.531219296 +0000 UTC m=+82.034905158" watchObservedRunningTime="2026-07-17 01:14:34.54929401 +0000 UTC m=+82.052979874"
Jul 17 01:14:34.598155 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.598096    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595cf9cf64-zkzc8" podStartSLOduration=3.282206942 podStartE2EDuration="10.597325046s" totalImagesPullingTime="7.315118104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:34.574677109 +0000 UTC m=+82.078362973" watchObservedRunningTime="2026-07-17 01:14:34.597325046 +0000 UTC m=+82.101010911"
Jul 17 01:14:34.622610 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:34.622554    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-cttwl" podStartSLOduration=2.308757502 podStartE2EDuration="7.622533376s" totalImagesPullingTime="5.313775874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:34.620880891 +0000 UTC m=+82.124566754" watchObservedRunningTime="2026-07-17 01:14:34.622533376 +0000 UTC m=+82.126219237"
Jul 17 01:14:38.475712 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:38.475675    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ks5wl"
Jul 17 01:14:39.535089 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:39.535039    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xtx4c" event={"ID":"6573de58-3849-4424-a40d-52deb60da3cb","Type":"ContainerStarted","Data":"b34d7195b44c73fcb10c8a2a8ad0ca5ed82aaef67c29caab0b7dc0647a0187e8"}
Jul 17 01:14:42.059528 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:42.059470    2582 patch_prober.go:28] interesting pod/image-registry-65f8df49d6-jl8sm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 17 01:14:42.059870 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:42.059581    2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm" podUID="40a719e1-db20-4055-83a8-f522d628bf9a" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 01:14:42.552261 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:42.552223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xtx4c" event={"ID":"6573de58-3849-4424-a40d-52deb60da3cb","Type":"ContainerStarted","Data":"0936c0f51c4df51dea2be195099d32a08f111125339d43c8358489216b5b1c8b"}
Jul 17 01:14:44.721966 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.721347    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xz72l"]
Jul 17 01:14:44.731920 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.731891    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.734601 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.734573    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k8bd4\""
Jul 17 01:14:44.734741 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.734652    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 01:14:44.734796 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.734787    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 01:14:44.734876 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.734860    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 01:14:44.735657 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.735628    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.777692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-wtmp\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.777742    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-tls\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.777779    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.777828    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdm98\" (UniqueName: \"kubernetes.io/projected/dac01c22-87b3-48bc-8bd9-53978f872b7b-kube-api-access-zdm98\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.777864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-textfile\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.777889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.777957    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-root\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.777978    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dac01c22-87b3-48bc-8bd9-53978f872b7b-metrics-client-ca\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.778142 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.778004    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-sys\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.878653 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878610    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-root\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.878848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dac01c22-87b3-48bc-8bd9-53978f872b7b-metrics-client-ca\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.878848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878694    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-sys\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.878848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878732    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-wtmp\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.878848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878739    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-root\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.878848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878762    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-tls\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.878848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.878848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878805    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-sys\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.879250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdm98\" (UniqueName: \"kubernetes.io/projected/dac01c22-87b3-48bc-8bd9-53978f872b7b-kube-api-access-zdm98\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.879250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-textfile\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.879250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.878935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.879732 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.879687    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-wtmp\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.879886 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.879838    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-textfile\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.880199 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.880179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dac01c22-87b3-48bc-8bd9-53978f872b7b-metrics-client-ca\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.880546 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.880520    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-accelerators-collector-config\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.882617 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.882591    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-tls\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.884236 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.884212    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dac01c22-87b3-48bc-8bd9-53978f872b7b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:44.887534 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:44.887481    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdm98\" (UniqueName: \"kubernetes.io/projected/dac01c22-87b3-48bc-8bd9-53978f872b7b-kube-api-access-zdm98\") pod \"node-exporter-xz72l\" (UID: \"dac01c22-87b3-48bc-8bd9-53978f872b7b\") " pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:45.044692 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:45.044609    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xz72l"
Jul 17 01:14:45.427577 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:45.425956    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:14:45.427577 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:45.427480    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:14:45.981800 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:14:45.981771    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddac01c22_87b3_48bc_8bd9_53978f872b7b.slice/crio-10046603632c140b610b6454228e37d08d204d273894d1ba4d87dc4a51bebfaa WatchSource:0}: Error finding container 10046603632c140b610b6454228e37d08d204d273894d1ba4d87dc4a51bebfaa: Status 404 returned error can't find the container with id 10046603632c140b610b6454228e37d08d204d273894d1ba4d87dc4a51bebfaa
Jul 17 01:14:46.571784 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:46.571751    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xtx4c" event={"ID":"6573de58-3849-4424-a40d-52deb60da3cb","Type":"ContainerStarted","Data":"f78ce12a08b0901471a0f384f385aa9d7063b25aef07e80c367ef7fbe4bfe319"}
Jul 17 01:14:46.573083 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:46.573056    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xz72l" event={"ID":"dac01c22-87b3-48bc-8bd9-53978f872b7b","Type":"ContainerStarted","Data":"10046603632c140b610b6454228e37d08d204d273894d1ba4d87dc4a51bebfaa"}
Jul 17 01:14:46.590880 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:46.590808    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xtx4c" podStartSLOduration=34.386603771 podStartE2EDuration="46.59078805s" totalImagesPullingTime="12.204184279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:14:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:46.59013285 +0000 UTC m=+94.093818715" watchObservedRunningTime="2026-07-17 01:14:46.59078805 +0000 UTC m=+94.094473915"
Jul 17 01:14:47.579074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:47.579037    2582 generic.go:356] "Generic (PLEG): container finished" podID="dac01c22-87b3-48bc-8bd9-53978f872b7b" containerID="52c796e155810ce4cd39e1eabcbbb3231cff09ee1e3ce556090fd7599f7f50e5" exitCode=0
Jul 17 01:14:47.579481 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:47.579120    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xz72l" event={"ID":"dac01c22-87b3-48bc-8bd9-53978f872b7b","Type":"ContainerDied","Data":"52c796e155810ce4cd39e1eabcbbb3231cff09ee1e3ce556090fd7599f7f50e5"}
Jul 17 01:14:48.585202 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:48.585166    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xz72l" event={"ID":"dac01c22-87b3-48bc-8bd9-53978f872b7b","Type":"ContainerStarted","Data":"f090b66fe4935b09b659d5357612c4f7ccac7607216467f39108faece857c3a9"}
Jul 17 01:14:48.585676 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:48.585220    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xz72l" event={"ID":"dac01c22-87b3-48bc-8bd9-53978f872b7b","Type":"ContainerStarted","Data":"63088950bdd31d8ef91529c330d838e2d11ebaafb140510d44acfcaea30147a7"}
Jul 17 01:14:48.608632 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:48.608577    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xz72l" podStartSLOduration=3.415245538 podStartE2EDuration="4.608559848s" totalImagesPullingTime="1.11976285s" totalInitContainerRuntime="73.55146ms" isStatefulPod=true podCreationTimestamp="2026-07-17 01:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:48.606770434 +0000 UTC m=+96.110456308" watchObservedRunningTime="2026-07-17 01:14:48.608559848 +0000 UTC m=+96.112245709"
Jul 17 01:14:50.362648 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.362619    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sshtq"
Jul 17 01:14:50.447299 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.447259    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq" podUID="020ed824-8ef0-49a1-bf9f-9f184d9bd086" containerName="registry" containerID="cri-o://3e4f8e047a505ed4fa2340915cd8d16080e2d22d21c121c780881ed6dfe0b902" gracePeriod=30
Jul 17 01:14:50.595647 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.595611    2582 generic.go:356] "Generic (PLEG): container finished" podID="020ed824-8ef0-49a1-bf9f-9f184d9bd086" containerID="3e4f8e047a505ed4fa2340915cd8d16080e2d22d21c121c780881ed6dfe0b902" exitCode=0
Jul 17 01:14:50.595810 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.595673    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq" event={"ID":"020ed824-8ef0-49a1-bf9f-9f184d9bd086","Type":"ContainerDied","Data":"3e4f8e047a505ed4fa2340915cd8d16080e2d22d21c121c780881ed6dfe0b902"}
Jul 17 01:14:50.697340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.697316    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:14:50.833639 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.833604    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-certificates\") pod \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") "
Jul 17 01:14:50.833831 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.833660    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-image-registry-private-configuration\") pod \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") "
Jul 17 01:14:50.833831 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.833710    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-trusted-ca\") pod \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") "
Jul 17 01:14:50.833831 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.833733    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-installation-pull-secrets\") pod \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") "
Jul 17 01:14:50.833831 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.833776    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/020ed824-8ef0-49a1-bf9f-9f184d9bd086-ca-trust-extracted\") pod \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") "
Jul 17 01:14:50.833831 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.833809    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkfhv\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-kube-api-access-nkfhv\") pod \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") "
Jul 17 01:14:50.834084 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.833837    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") pod \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") "
Jul 17 01:14:50.834084 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.833866    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-bound-sa-token\") pod \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\" (UID: \"020ed824-8ef0-49a1-bf9f-9f184d9bd086\") "
Jul 17 01:14:50.834229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.834101    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "020ed824-8ef0-49a1-bf9f-9f184d9bd086" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:14:50.834315 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.834225    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "020ed824-8ef0-49a1-bf9f-9f184d9bd086" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:14:50.836446 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.836418    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "020ed824-8ef0-49a1-bf9f-9f184d9bd086" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 01:14:50.836649 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.836611    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "020ed824-8ef0-49a1-bf9f-9f184d9bd086" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 01:14:50.836835 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.836812    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "020ed824-8ef0-49a1-bf9f-9f184d9bd086" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 01:14:50.836904 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.836886    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-kube-api-access-nkfhv" (OuterVolumeSpecName: "kube-api-access-nkfhv") pod "020ed824-8ef0-49a1-bf9f-9f184d9bd086" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086"). InnerVolumeSpecName "kube-api-access-nkfhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 01:14:50.836978 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.836956    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "020ed824-8ef0-49a1-bf9f-9f184d9bd086" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 01:14:50.843165 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.843142    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/020ed824-8ef0-49a1-bf9f-9f184d9bd086-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "020ed824-8ef0-49a1-bf9f-9f184d9bd086" (UID: "020ed824-8ef0-49a1-bf9f-9f184d9bd086"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 01:14:50.934754 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.934723    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-certificates\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:14:50.934754 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.934759    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-image-registry-private-configuration\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:14:50.934944 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.934769    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/020ed824-8ef0-49a1-bf9f-9f184d9bd086-trusted-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:14:50.934944 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.934779    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/020ed824-8ef0-49a1-bf9f-9f184d9bd086-installation-pull-secrets\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:14:50.934944 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.934788    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/020ed824-8ef0-49a1-bf9f-9f184d9bd086-ca-trust-extracted\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:14:50.934944 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.934796    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkfhv\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-kube-api-access-nkfhv\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:14:50.934944 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.934804    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-registry-tls\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:14:50.934944 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:50.934814    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/020ed824-8ef0-49a1-bf9f-9f184d9bd086-bound-sa-token\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:14:51.414614 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:51.414530    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65f8df49d6-jl8sm"]
Jul 17 01:14:51.601162 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:51.601134    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq"
Jul 17 01:14:51.601162 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:51.601144    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78d7f6fd5-j9xgq" event={"ID":"020ed824-8ef0-49a1-bf9f-9f184d9bd086","Type":"ContainerDied","Data":"a504269cb1ef5ff939951db25f2915c97943beb64d1e39cd94aa637ddf7d5065"}
Jul 17 01:14:51.601388 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:51.601199    2582 scope.go:117] "RemoveContainer" containerID="3e4f8e047a505ed4fa2340915cd8d16080e2d22d21c121c780881ed6dfe0b902"
Jul 17 01:14:51.616242 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:51.616199    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78d7f6fd5-j9xgq"]
Jul 17 01:14:51.621816 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:51.621791    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78d7f6fd5-j9xgq"]
Jul 17 01:14:52.989835 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:14:52.989798    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="020ed824-8ef0-49a1-bf9f-9f184d9bd086" path="/var/lib/kubelet/pods/020ed824-8ef0-49a1-bf9f-9f184d9bd086/volumes"
Jul 17 01:15:06.659037 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:06.658998    2582 generic.go:356] "Generic (PLEG): container finished" podID="438d40b6-c5d1-44e6-b796-c5f6ec836252" containerID="9c47730dfcef26679b1dffd3538156a6dee1099c890368a8218ffc10b8c012eb" exitCode=0
Jul 17 01:15:06.659533 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:06.659047    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j" event={"ID":"438d40b6-c5d1-44e6-b796-c5f6ec836252","Type":"ContainerDied","Data":"9c47730dfcef26679b1dffd3538156a6dee1099c890368a8218ffc10b8c012eb"}
Jul 17 01:15:06.659533 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:06.659344    2582 scope.go:117] "RemoveContainer" containerID="9c47730dfcef26679b1dffd3538156a6dee1099c890368a8218ffc10b8c012eb"
Jul 17 01:15:07.665259 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:07.665217    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-4wn4j" event={"ID":"438d40b6-c5d1-44e6-b796-c5f6ec836252","Type":"ContainerStarted","Data":"c359a8fbaa2f46ce76235b182009b75edcafaa64406877a622876927043a018a"}
Jul 17 01:15:14.695274 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:14.695241    2582 generic.go:356] "Generic (PLEG): container finished" podID="5b704339-44a2-4e1c-96b1-3950ca6a5d13" containerID="cee53458eaadf8c796d4d23370914adf9fb3648d54a44154e53b5bb2b44da271" exitCode=0
Jul 17 01:15:14.695813 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:14.695321    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj" event={"ID":"5b704339-44a2-4e1c-96b1-3950ca6a5d13","Type":"ContainerDied","Data":"cee53458eaadf8c796d4d23370914adf9fb3648d54a44154e53b5bb2b44da271"}
Jul 17 01:15:14.695887 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:14.695830    2582 scope.go:117] "RemoveContainer" containerID="cee53458eaadf8c796d4d23370914adf9fb3648d54a44154e53b5bb2b44da271"
Jul 17 01:15:15.701528 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:15.701466    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-9xsvj" event={"ID":"5b704339-44a2-4e1c-96b1-3950ca6a5d13","Type":"ContainerStarted","Data":"0f1d5ccf405fde84c031fd1e9011b3c8a7448ccfe80d665a95bb33b5224581b1"}
Jul 17 01:15:16.435031 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.434976    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm" podUID="40a719e1-db20-4055-83a8-f522d628bf9a" containerName="registry" containerID="cri-o://c5923554da6d979c82367700ac9ed58eb0481ce1755052de64764c4593e02078" gracePeriod=30
Jul 17 01:15:16.706129 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.706043    2582 generic.go:356] "Generic (PLEG): container finished" podID="40a719e1-db20-4055-83a8-f522d628bf9a" containerID="c5923554da6d979c82367700ac9ed58eb0481ce1755052de64764c4593e02078" exitCode=0
Jul 17 01:15:16.706476 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.706130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm" event={"ID":"40a719e1-db20-4055-83a8-f522d628bf9a","Type":"ContainerDied","Data":"c5923554da6d979c82367700ac9ed58eb0481ce1755052de64764c4593e02078"}
Jul 17 01:15:16.706476 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.706190    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm" event={"ID":"40a719e1-db20-4055-83a8-f522d628bf9a","Type":"ContainerDied","Data":"0f8f9e222740e6430970b425bc388039d063597335892ae8ad8d89577ec71ac9"}
Jul 17 01:15:16.706476 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.706208    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f8f9e222740e6430970b425bc388039d063597335892ae8ad8d89577ec71ac9"
Jul 17 01:15:16.713016 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.712993    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:15:16.745667 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.745635    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-bound-sa-token\") pod \"40a719e1-db20-4055-83a8-f522d628bf9a\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") "
Jul 17 01:15:16.745851 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.745674    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-installation-pull-secrets\") pod \"40a719e1-db20-4055-83a8-f522d628bf9a\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") "
Jul 17 01:15:16.745851 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.745698    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/40a719e1-db20-4055-83a8-f522d628bf9a-ca-trust-extracted\") pod \"40a719e1-db20-4055-83a8-f522d628bf9a\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") "
Jul 17 01:15:16.745851 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.745715    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") pod \"40a719e1-db20-4055-83a8-f522d628bf9a\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") "
Jul 17 01:15:16.745851 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.745756    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-trusted-ca\") pod \"40a719e1-db20-4055-83a8-f522d628bf9a\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") "
Jul 17 01:15:16.745851 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.745777    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-registry-certificates\") pod \"40a719e1-db20-4055-83a8-f522d628bf9a\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") "
Jul 17 01:15:16.745851 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.745795    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvvmh\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-kube-api-access-pvvmh\") pod \"40a719e1-db20-4055-83a8-f522d628bf9a\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") "
Jul 17 01:15:16.745851 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.745823    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-image-registry-private-configuration\") pod \"40a719e1-db20-4055-83a8-f522d628bf9a\" (UID: \"40a719e1-db20-4055-83a8-f522d628bf9a\") "
Jul 17 01:15:16.746687 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.746659    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "40a719e1-db20-4055-83a8-f522d628bf9a" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:15:16.746852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.746821    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "40a719e1-db20-4055-83a8-f522d628bf9a" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:15:16.748960 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.748917    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "40a719e1-db20-4055-83a8-f522d628bf9a" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 01:15:16.749066 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.748989    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "40a719e1-db20-4055-83a8-f522d628bf9a" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 01:15:16.749066 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.749031    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "40a719e1-db20-4055-83a8-f522d628bf9a" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 01:15:16.749155 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.749061    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "40a719e1-db20-4055-83a8-f522d628bf9a" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 01:15:16.749456 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.749433    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-kube-api-access-pvvmh" (OuterVolumeSpecName: "kube-api-access-pvvmh") pod "40a719e1-db20-4055-83a8-f522d628bf9a" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a"). InnerVolumeSpecName "kube-api-access-pvvmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 01:15:16.756064 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.756033    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a719e1-db20-4055-83a8-f522d628bf9a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "40a719e1-db20-4055-83a8-f522d628bf9a" (UID: "40a719e1-db20-4055-83a8-f522d628bf9a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 01:15:16.847744 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.847706    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-bound-sa-token\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:15:16.847744 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.847738    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-installation-pull-secrets\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:15:16.847744 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.847748    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/40a719e1-db20-4055-83a8-f522d628bf9a-ca-trust-extracted\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:15:16.847744 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.847757    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-registry-tls\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:15:16.848004 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.847767    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-trusted-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:15:16.848004 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.847775    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/40a719e1-db20-4055-83a8-f522d628bf9a-registry-certificates\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:15:16.848004 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.847783    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvvmh\" (UniqueName: \"kubernetes.io/projected/40a719e1-db20-4055-83a8-f522d628bf9a-kube-api-access-pvvmh\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:15:16.848004 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:16.847792    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/40a719e1-db20-4055-83a8-f522d628bf9a-image-registry-private-configuration\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:15:17.711843 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:17.711795    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f8df49d6-jl8sm"
Jul 17 01:15:17.731456 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:17.731423    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65f8df49d6-jl8sm"]
Jul 17 01:15:17.734510 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:17.734461    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65f8df49d6-jl8sm"]
Jul 17 01:15:18.989332 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:18.989295    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40a719e1-db20-4055-83a8-f522d628bf9a" path="/var/lib/kubelet/pods/40a719e1-db20-4055-83a8-f522d628bf9a/volumes"
Jul 17 01:15:25.749789 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:25.749756    2582 generic.go:356] "Generic (PLEG): container finished" podID="32a81af9-c586-4ec0-b2a1-dca8d44df4fd" containerID="783d8c2f2ffbd8668882c96d27c160057ba3847a86cd5c2d39d8c040ba3c7cf7" exitCode=0
Jul 17 01:15:25.750162 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:25.749819    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-l9lkq" event={"ID":"32a81af9-c586-4ec0-b2a1-dca8d44df4fd","Type":"ContainerDied","Data":"783d8c2f2ffbd8668882c96d27c160057ba3847a86cd5c2d39d8c040ba3c7cf7"}
Jul 17 01:15:25.750162 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:25.750156    2582 scope.go:117] "RemoveContainer" containerID="783d8c2f2ffbd8668882c96d27c160057ba3847a86cd5c2d39d8c040ba3c7cf7"
Jul 17 01:15:26.756133 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:15:26.756099    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-l9lkq" event={"ID":"32a81af9-c586-4ec0-b2a1-dca8d44df4fd","Type":"ContainerStarted","Data":"c20b37e74b47517f1d2bd606179b3edc88a6d3ae0b3c733aebbb4a489ed2ee2a"}
Jul 17 01:17:38.606345 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.606261    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"]
Jul 17 01:17:38.606830 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.606589    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="020ed824-8ef0-49a1-bf9f-9f184d9bd086" containerName="registry"
Jul 17 01:17:38.606830 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.606602    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="020ed824-8ef0-49a1-bf9f-9f184d9bd086" containerName="registry"
Jul 17 01:17:38.606830 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.606632    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a719e1-db20-4055-83a8-f522d628bf9a" containerName="registry"
Jul 17 01:17:38.606830 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.606638    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a719e1-db20-4055-83a8-f522d628bf9a" containerName="registry"
Jul 17 01:17:38.606830 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.606693    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="020ed824-8ef0-49a1-bf9f-9f184d9bd086" containerName="registry"
Jul 17 01:17:38.606830 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.606707    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="40a719e1-db20-4055-83a8-f522d628bf9a" containerName="registry"
Jul 17 01:17:38.609686 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.609668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.611899 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.611869    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 01:17:38.612028 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.611976    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-276j7\""
Jul 17 01:17:38.612097 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.612082    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 01:17:38.612174 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.612159    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 01:17:38.612207 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.612181    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 01:17:38.629043 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.629013    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"]
Jul 17 01:17:38.703710 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.703670    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faf3187d-a6c7-4550-90c2-843b37a09f8f-tmp\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.703889 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.703711    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/faf3187d-a6c7-4550-90c2-843b37a09f8f-webhook-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.703889 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.703791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lctf\" (UniqueName: \"kubernetes.io/projected/faf3187d-a6c7-4550-90c2-843b37a09f8f-kube-api-access-8lctf\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.703889 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.703845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/faf3187d-a6c7-4550-90c2-843b37a09f8f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.703889 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.703886    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/faf3187d-a6c7-4550-90c2-843b37a09f8f-manifests\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.805091 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.805057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/faf3187d-a6c7-4550-90c2-843b37a09f8f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.805278 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.805104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/faf3187d-a6c7-4550-90c2-843b37a09f8f-manifests\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.805278 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.805135    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faf3187d-a6c7-4550-90c2-843b37a09f8f-tmp\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.805278 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.805162    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/faf3187d-a6c7-4550-90c2-843b37a09f8f-webhook-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.805278 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.805230    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lctf\" (UniqueName: \"kubernetes.io/projected/faf3187d-a6c7-4550-90c2-843b37a09f8f-kube-api-access-8lctf\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.805607 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.805567    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/faf3187d-a6c7-4550-90c2-843b37a09f8f-manifests\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.807764 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.807730    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/faf3187d-a6c7-4550-90c2-843b37a09f8f-tmp\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.807764 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.807737    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/faf3187d-a6c7-4550-90c2-843b37a09f8f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.807922 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.807895    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/faf3187d-a6c7-4550-90c2-843b37a09f8f-webhook-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.813426 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.813404    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lctf\" (UniqueName: \"kubernetes.io/projected/faf3187d-a6c7-4550-90c2-843b37a09f8f-kube-api-access-8lctf\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5\" (UID: \"faf3187d-a6c7-4550-90c2-843b37a09f8f\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:38.919812 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:38.919717    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:39.063025 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:39.062952    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"]
Jul 17 01:17:39.066233 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:17:39.066208    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf3187d_a6c7_4550_90c2_843b37a09f8f.slice/crio-24d24af586c28a6682f1e960d3bab9e812d712dbfd20edcd0c4418fc54bcae02 WatchSource:0}: Error finding container 24d24af586c28a6682f1e960d3bab9e812d712dbfd20edcd0c4418fc54bcae02: Status 404 returned error can't find the container with id 24d24af586c28a6682f1e960d3bab9e812d712dbfd20edcd0c4418fc54bcae02
Jul 17 01:17:39.291914 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:39.291882    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5" event={"ID":"faf3187d-a6c7-4550-90c2-843b37a09f8f","Type":"ContainerStarted","Data":"24d24af586c28a6682f1e960d3bab9e812d712dbfd20edcd0c4418fc54bcae02"}
Jul 17 01:17:42.307450 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:42.307345    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5" event={"ID":"faf3187d-a6c7-4550-90c2-843b37a09f8f","Type":"ContainerStarted","Data":"6a9e7f0f49e1f761cd2e058d03c9ae99f613f5507ec2e7039f5d47d86aca0450"}
Jul 17 01:17:43.313591 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:43.313554    2582 generic.go:356] "Generic (PLEG): container finished" podID="faf3187d-a6c7-4550-90c2-843b37a09f8f" containerID="6a9e7f0f49e1f761cd2e058d03c9ae99f613f5507ec2e7039f5d47d86aca0450" exitCode=0
Jul 17 01:17:43.313991 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:43.313639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5" event={"ID":"faf3187d-a6c7-4550-90c2-843b37a09f8f","Type":"ContainerDied","Data":"6a9e7f0f49e1f761cd2e058d03c9ae99f613f5507ec2e7039f5d47d86aca0450"}
Jul 17 01:17:44.320669 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:44.320629    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5" event={"ID":"faf3187d-a6c7-4550-90c2-843b37a09f8f","Type":"ContainerStarted","Data":"c5ac6c754811bd0a530a15199c216cf52449d7d77282d3b8fa880ef89f544bb5"}
Jul 17 01:17:44.321146 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:44.320773    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:17:44.346835 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:44.346773    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5" podStartSLOduration=1.887281712 podStartE2EDuration="6.34675052s" totalImagesPullingTime="3.318056605s" totalInitContainerRuntime="1.141412203s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:17:44.34446418 +0000 UTC m=+271.848150041" watchObservedRunningTime="2026-07-17 01:17:44.34675052 +0000 UTC m=+271.850436383"
Jul 17 01:17:55.327610 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:17:55.327579    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5"
Jul 17 01:18:06.255628 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.255592    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"]
Jul 17 01:18:06.259232 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.259210    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.261713 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.261689    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 01:18:06.261838 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.261786    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7zm8p\""
Jul 17 01:18:06.279194 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.279168    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"]
Jul 17 01:18:06.352119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352018    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.352119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352086    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.352119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352114    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.352402 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352160    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.352402 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352186    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.352402 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352290    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.352402 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352333    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tdwx\" (UniqueName: \"kubernetes.io/projected/eff98150-13ba-4bc0-a104-6ff7b4a177d3-kube-api-access-6tdwx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.352402 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352388    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.352605 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.352411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.453834 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.453787    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454012 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.453841    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tdwx\" (UniqueName: \"kubernetes.io/projected/eff98150-13ba-4bc0-a104-6ff7b4a177d3-kube-api-access-6tdwx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454012 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.453889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454012 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.453913    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454012 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.453952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454012 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.453998    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454279 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.454026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454279 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.454077    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454279 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.454104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454428 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.454417    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454537 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.454479    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454697 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.454671    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454855 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.454764    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.454950 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.454789    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.456765 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.456745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.456869 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.456783    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.467576 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.467553    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tdwx\" (UniqueName: \"kubernetes.io/projected/eff98150-13ba-4bc0-a104-6ff7b4a177d3-kube-api-access-6tdwx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.469236 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.469215    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff98150-13ba-4bc0-a104-6ff7b4a177d3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mm6t5\" (UID: \"eff98150-13ba-4bc0-a104-6ff7b4a177d3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.570662 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.570565    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:06.725268 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:06.725244    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"]
Jul 17 01:18:06.727795 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:18:06.727767    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff98150_13ba_4bc0_a104_6ff7b4a177d3.slice/crio-bb96e992150e5cf05a1030eb874fd586b534f7d1cbee856c3e5672044606bd7a WatchSource:0}: Error finding container bb96e992150e5cf05a1030eb874fd586b534f7d1cbee856c3e5672044606bd7a: Status 404 returned error can't find the container with id bb96e992150e5cf05a1030eb874fd586b534f7d1cbee856c3e5672044606bd7a
Jul 17 01:18:07.422432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:07.422394    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5" event={"ID":"eff98150-13ba-4bc0-a104-6ff7b4a177d3","Type":"ContainerStarted","Data":"bb96e992150e5cf05a1030eb874fd586b534f7d1cbee856c3e5672044606bd7a"}
Jul 17 01:18:08.571569 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.571478    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"]
Jul 17 01:18:08.575442 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.575416    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:08.578752 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.578597    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 17 01:18:08.578752 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.578646    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vrhct\""
Jul 17 01:18:08.588358 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.587982    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"]
Jul 17 01:18:08.590423 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.590407    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"]
Jul 17 01:18:08.598115 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.596408    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:08.599057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.599037    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 17 01:18:08.599366 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.599346    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t5f27\""
Jul 17 01:18:08.605236 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.605214    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"]
Jul 17 01:18:08.675424 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.675387    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6989d7c1-045d-450d-b55e-f1c329c1c59c-cert\") pod \"kserve-controller-manager-d6f4b65b8-8fc9m\" (UID: \"6989d7c1-045d-450d-b55e-f1c329c1c59c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:08.675603 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.675440    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n97lq\" (UniqueName: \"kubernetes.io/projected/6989d7c1-045d-450d-b55e-f1c329c1c59c-kube-api-access-n97lq\") pod \"kserve-controller-manager-d6f4b65b8-8fc9m\" (UID: \"6989d7c1-045d-450d-b55e-f1c329c1c59c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:08.776429 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.776396    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csgkb\" (UniqueName: \"kubernetes.io/projected/951173fb-ea92-4a52-b7a5-b5453d9d16b0-kube-api-access-csgkb\") pod \"llmisvc-controller-manager-777c6bd7d5-chljj\" (UID: \"951173fb-ea92-4a52-b7a5-b5453d9d16b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:08.776685 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.776445    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/951173fb-ea92-4a52-b7a5-b5453d9d16b0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-chljj\" (UID: \"951173fb-ea92-4a52-b7a5-b5453d9d16b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:08.776685 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.776517    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6989d7c1-045d-450d-b55e-f1c329c1c59c-cert\") pod \"kserve-controller-manager-d6f4b65b8-8fc9m\" (UID: \"6989d7c1-045d-450d-b55e-f1c329c1c59c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:08.776685 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.776544    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n97lq\" (UniqueName: \"kubernetes.io/projected/6989d7c1-045d-450d-b55e-f1c329c1c59c-kube-api-access-n97lq\") pod \"kserve-controller-manager-d6f4b65b8-8fc9m\" (UID: \"6989d7c1-045d-450d-b55e-f1c329c1c59c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:08.779340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.779289    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6989d7c1-045d-450d-b55e-f1c329c1c59c-cert\") pod \"kserve-controller-manager-d6f4b65b8-8fc9m\" (UID: \"6989d7c1-045d-450d-b55e-f1c329c1c59c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:08.788609 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.788558    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n97lq\" (UniqueName: \"kubernetes.io/projected/6989d7c1-045d-450d-b55e-f1c329c1c59c-kube-api-access-n97lq\") pod \"kserve-controller-manager-d6f4b65b8-8fc9m\" (UID: \"6989d7c1-045d-450d-b55e-f1c329c1c59c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:08.877706 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.877614    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csgkb\" (UniqueName: \"kubernetes.io/projected/951173fb-ea92-4a52-b7a5-b5453d9d16b0-kube-api-access-csgkb\") pod \"llmisvc-controller-manager-777c6bd7d5-chljj\" (UID: \"951173fb-ea92-4a52-b7a5-b5453d9d16b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:08.877706 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.877674    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/951173fb-ea92-4a52-b7a5-b5453d9d16b0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-chljj\" (UID: \"951173fb-ea92-4a52-b7a5-b5453d9d16b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:08.880624 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.880599    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/951173fb-ea92-4a52-b7a5-b5453d9d16b0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-chljj\" (UID: \"951173fb-ea92-4a52-b7a5-b5453d9d16b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:08.893550 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.893518    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:08.900654 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.900298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csgkb\" (UniqueName: \"kubernetes.io/projected/951173fb-ea92-4a52-b7a5-b5453d9d16b0-kube-api-access-csgkb\") pod \"llmisvc-controller-manager-777c6bd7d5-chljj\" (UID: \"951173fb-ea92-4a52-b7a5-b5453d9d16b0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:08.915428 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:08.915402    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:09.225483 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.225445    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"]
Jul 17 01:18:09.229185 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.229153    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.231928 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.231897    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 17 01:18:09.232790 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.232761    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 17 01:18:09.232790 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.232761    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 17 01:18:09.232976 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.232796    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 17 01:18:09.232976 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.232761    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 17 01:18:09.232976 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.232800    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hz2wm\""
Jul 17 01:18:09.238555 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.238530    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"]
Jul 17 01:18:09.282450 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.282408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/deff0458-dd21-45ee-937a-f0b830492acc-manager-config\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.282450 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.282453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/deff0458-dd21-45ee-937a-f0b830492acc-metrics-cert\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.282712 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.282586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g79w8\" (UniqueName: \"kubernetes.io/projected/deff0458-dd21-45ee-937a-f0b830492acc-kube-api-access-g79w8\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.282712 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.282668    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deff0458-dd21-45ee-937a-f0b830492acc-cert\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.383842 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.383802    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/deff0458-dd21-45ee-937a-f0b830492acc-manager-config\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.383842 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.383851    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/deff0458-dd21-45ee-937a-f0b830492acc-metrics-cert\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.384150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.383927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g79w8\" (UniqueName: \"kubernetes.io/projected/deff0458-dd21-45ee-937a-f0b830492acc-kube-api-access-g79w8\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.384150 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.383995    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deff0458-dd21-45ee-937a-f0b830492acc-cert\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.384601 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.384577    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/deff0458-dd21-45ee-937a-f0b830492acc-manager-config\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.386923 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.386898    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/deff0458-dd21-45ee-937a-f0b830492acc-metrics-cert\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.387093 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.387074    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deff0458-dd21-45ee-937a-f0b830492acc-cert\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.397367 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.397344    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g79w8\" (UniqueName: \"kubernetes.io/projected/deff0458-dd21-45ee-937a-f0b830492acc-kube-api-access-g79w8\") pod \"lws-controller-manager-66ffdbf46d-gn45x\" (UID: \"deff0458-dd21-45ee-937a-f0b830492acc\") " pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.541175 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.541073    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:09.757580 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.757530    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:18:09.758135 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.757640    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:18:09.758135 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.757683    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:18:09.851455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:09.851377    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"]
Jul 17 01:18:09.853889 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:18:09.853860    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeff0458_dd21_45ee_937a_f0b830492acc.slice/crio-a02f7efefbaec4545e816c9fb312f16318d1cf64b45b822fa1b11b218c1b385b WatchSource:0}: Error finding container a02f7efefbaec4545e816c9fb312f16318d1cf64b45b822fa1b11b218c1b385b: Status 404 returned error can't find the container with id a02f7efefbaec4545e816c9fb312f16318d1cf64b45b822fa1b11b218c1b385b
Jul 17 01:18:10.068674 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.068586    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"]
Jul 17 01:18:10.070937 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.070913    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"]
Jul 17 01:18:10.072397 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:18:10.072367    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6989d7c1_045d_450d_b55e_f1c329c1c59c.slice/crio-68dfb800bf056262ee71014ec0e8720a439a29d76bc31313fa29d03e1bb63c6a WatchSource:0}: Error finding container 68dfb800bf056262ee71014ec0e8720a439a29d76bc31313fa29d03e1bb63c6a: Status 404 returned error can't find the container with id 68dfb800bf056262ee71014ec0e8720a439a29d76bc31313fa29d03e1bb63c6a
Jul 17 01:18:10.073998 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:18:10.073966    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod951173fb_ea92_4a52_b7a5_b5453d9d16b0.slice/crio-7bf9b9691e5cf1beab682729769ea17457e05f29da51042ffdd6eeaf0d27ca99 WatchSource:0}: Error finding container 7bf9b9691e5cf1beab682729769ea17457e05f29da51042ffdd6eeaf0d27ca99: Status 404 returned error can't find the container with id 7bf9b9691e5cf1beab682729769ea17457e05f29da51042ffdd6eeaf0d27ca99
Jul 17 01:18:10.440920 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.440886    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x" event={"ID":"deff0458-dd21-45ee-937a-f0b830492acc","Type":"ContainerStarted","Data":"a02f7efefbaec4545e816c9fb312f16318d1cf64b45b822fa1b11b218c1b385b"}
Jul 17 01:18:10.442046 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.442021    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m" event={"ID":"6989d7c1-045d-450d-b55e-f1c329c1c59c","Type":"ContainerStarted","Data":"68dfb800bf056262ee71014ec0e8720a439a29d76bc31313fa29d03e1bb63c6a"}
Jul 17 01:18:10.443066 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.443045    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj" event={"ID":"951173fb-ea92-4a52-b7a5-b5453d9d16b0","Type":"ContainerStarted","Data":"7bf9b9691e5cf1beab682729769ea17457e05f29da51042ffdd6eeaf0d27ca99"}
Jul 17 01:18:10.444377 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.444357    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5" event={"ID":"eff98150-13ba-4bc0-a104-6ff7b4a177d3","Type":"ContainerStarted","Data":"d3c67fec7c800488e1586b28a23a0d7d4667705e10259cef8c2e15fe789fa5a5"}
Jul 17 01:18:10.490782 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.490720    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5" podStartSLOduration=1.463251042 podStartE2EDuration="4.490698266s" totalImagesPullingTime="3.027447224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:18:10.488726059 +0000 UTC m=+297.992411922" watchObservedRunningTime="2026-07-17 01:18:10.490698266 +0000 UTC m=+297.994384131"
Jul 17 01:18:10.571087 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.571051    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:10.576222 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:10.576197    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:11.450369 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:11.450298    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:11.451465 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:11.451439    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mm6t5"
Jul 17 01:18:14.150750 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.150724    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:18:14.151157 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.150894    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:18:14.158634 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.158610    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:18:14.159208 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.159189    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:18:14.162128 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.162109    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 01:18:14.467545 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.467476    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x" event={"ID":"deff0458-dd21-45ee-937a-f0b830492acc","Type":"ContainerStarted","Data":"82c3c55329768d06114e5eda105566e64a0a6cb1a48847ae958a8ba4fa200363"}
Jul 17 01:18:14.467746 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.467612    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:14.469242 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.469214    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m" event={"ID":"6989d7c1-045d-450d-b55e-f1c329c1c59c","Type":"ContainerStarted","Data":"7cec1930d1e8cf5d213efc788349752054f72bfd53b5ce11eec06f4c7bb64333"}
Jul 17 01:18:14.469372 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.469272    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:18:14.470956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.470921    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj" event={"ID":"951173fb-ea92-4a52-b7a5-b5453d9d16b0","Type":"ContainerStarted","Data":"a5889187eb1f8765cd95cdd384a77f075a9cb1ebc54bb5fd9a7470714ded4f52"}
Jul 17 01:18:14.471082 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.471055    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:14.483992 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.483944    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x" podStartSLOduration=1.212055974 podStartE2EDuration="5.483928601s" totalImagesPullingTime="4.271872627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:18:14.48291995 +0000 UTC m=+301.986605811" watchObservedRunningTime="2026-07-17 01:18:14.483928601 +0000 UTC m=+301.987614462"
Jul 17 01:18:14.500138 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.500091    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m" podStartSLOduration=2.445888612 podStartE2EDuration="6.500074948s" totalImagesPullingTime="4.054186336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:18:14.498815767 +0000 UTC m=+302.002501630" watchObservedRunningTime="2026-07-17 01:18:14.500074948 +0000 UTC m=+302.003760809"
Jul 17 01:18:14.515005 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:14.514957    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj" podStartSLOduration=2.401876531 podStartE2EDuration="6.514940336s" totalImagesPullingTime="4.113063805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:18:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:18:14.51368365 +0000 UTC m=+302.017369515" watchObservedRunningTime="2026-07-17 01:18:14.514940336 +0000 UTC m=+302.018626199"
Jul 17 01:18:25.477696 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:25.477660    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66ffdbf46d-gn45x"
Jul 17 01:18:45.477950 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:45.477913    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-chljj"
Jul 17 01:18:45.481410 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:18:45.481391    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8fc9m"
Jul 17 01:20:55.645036 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.645004    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:20:55.648419 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.648398    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:55.650686 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.650658    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 01:20:55.650848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.650829    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 17 01:20:55.650917 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.650903    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 01:20:55.651636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.651621    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wmsvn\""
Jul 17 01:20:55.655807 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.655785    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:20:55.682000 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.681962    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:20:55.732733 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.732695    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/338ade41-85c7-47f4-b972-0fa46b25ee98-config-file\") pod \"limitador-limitador-54d6d988d5-qmzhb\" (UID: \"338ade41-85c7-47f4-b972-0fa46b25ee98\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:55.732911 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.732733    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzhvz\" (UniqueName: \"kubernetes.io/projected/338ade41-85c7-47f4-b972-0fa46b25ee98-kube-api-access-zzhvz\") pod \"limitador-limitador-54d6d988d5-qmzhb\" (UID: \"338ade41-85c7-47f4-b972-0fa46b25ee98\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:55.834202 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.834165    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/338ade41-85c7-47f4-b972-0fa46b25ee98-config-file\") pod \"limitador-limitador-54d6d988d5-qmzhb\" (UID: \"338ade41-85c7-47f4-b972-0fa46b25ee98\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:55.834404 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.834208    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzhvz\" (UniqueName: \"kubernetes.io/projected/338ade41-85c7-47f4-b972-0fa46b25ee98-kube-api-access-zzhvz\") pod \"limitador-limitador-54d6d988d5-qmzhb\" (UID: \"338ade41-85c7-47f4-b972-0fa46b25ee98\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:55.834887 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.834864    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/338ade41-85c7-47f4-b972-0fa46b25ee98-config-file\") pod \"limitador-limitador-54d6d988d5-qmzhb\" (UID: \"338ade41-85c7-47f4-b972-0fa46b25ee98\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:55.842432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.842410    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzhvz\" (UniqueName: \"kubernetes.io/projected/338ade41-85c7-47f4-b972-0fa46b25ee98-kube-api-access-zzhvz\") pod \"limitador-limitador-54d6d988d5-qmzhb\" (UID: \"338ade41-85c7-47f4-b972-0fa46b25ee98\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:55.959646 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:55.959608    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:56.097621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:56.097597    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:20:56.100427 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:20:56.100399    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338ade41_85c7_47f4_b972_0fa46b25ee98.slice/crio-718ef23a08a11e64ba8fd5f9f25cb14899389308b327c45878f00045f1cccbc4 WatchSource:0}: Error finding container 718ef23a08a11e64ba8fd5f9f25cb14899389308b327c45878f00045f1cccbc4: Status 404 returned error can't find the container with id 718ef23a08a11e64ba8fd5f9f25cb14899389308b327c45878f00045f1cccbc4
Jul 17 01:20:56.102763 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:56.102746    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 01:20:56.199553 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:56.199520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb" event={"ID":"338ade41-85c7-47f4-b972-0fa46b25ee98","Type":"ContainerStarted","Data":"718ef23a08a11e64ba8fd5f9f25cb14899389308b327c45878f00045f1cccbc4"}
Jul 17 01:20:59.216058 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:59.216011    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb" event={"ID":"338ade41-85c7-47f4-b972-0fa46b25ee98","Type":"ContainerStarted","Data":"5263551cd55fbf4e2ab1a3fbc1446377ddabeecd6a7646e15497c3ffc51ad9d8"}
Jul 17 01:20:59.216541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:59.216098    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:20:59.231847 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:20:59.231799    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb" podStartSLOduration=1.803784017 podStartE2EDuration="4.231784624s" totalImagesPullingTime="2.428000607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:20:59.230339695 +0000 UTC m=+466.734025562" watchObservedRunningTime="2026-07-17 01:20:59.231784624 +0000 UTC m=+466.735470486"
Jul 17 01:21:10.221203 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:21:10.221166    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"
Jul 17 01:21:12.953967 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:21:12.953935    2582 scope.go:117] "RemoveContainer" containerID="c5923554da6d979c82367700ac9ed58eb0481ce1755052de64764c4593e02078"
Jul 17 01:22:07.934958 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:07.934870    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:22:29.255780 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:29.255739    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:22:45.164639 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.164607    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"]
Jul 17 01:22:45.168264 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.168240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.170613 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.170585    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 01:22:45.171346 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.171326    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 01:22:45.171465 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.171379    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bjgtp\""
Jul 17 01:22:45.171576 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.171484    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 17 01:22:45.178070 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.178050    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"]
Jul 17 01:22:45.245708 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.245674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkbnt\" (UniqueName: \"kubernetes.io/projected/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-kube-api-access-fkbnt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.245873 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.245727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.245873 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.245751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.245873 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.245800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.245873 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.245830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.246027 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.245905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.346930 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.346890    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.346930 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.346927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.347179 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.346971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.347179 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.347003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkbnt\" (UniqueName: \"kubernetes.io/projected/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-kube-api-access-fkbnt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.347179 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.347138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.347287 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.347183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.347335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.347321    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.347377 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.347365    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.347559 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.347536    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.349362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.349345    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.349749 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.349730    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.354637 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.354619    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkbnt\" (UniqueName: \"kubernetes.io/projected/6956ac5e-0202-48c2-9f0c-e5be0e3014c4-kube-api-access-fkbnt\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qnn7t\" (UID: \"6956ac5e-0202-48c2-9f0c-e5be0e3014c4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.482669 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.482637    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:45.621891 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.621868    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"]
Jul 17 01:22:45.624385 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:22:45.624357    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6956ac5e_0202_48c2_9f0c_e5be0e3014c4.slice/crio-c3e709bda02a12dcb0e25058e0d6c3f20025f806e6a166dff2b4bde3a5237118 WatchSource:0}: Error finding container c3e709bda02a12dcb0e25058e0d6c3f20025f806e6a166dff2b4bde3a5237118: Status 404 returned error can't find the container with id c3e709bda02a12dcb0e25058e0d6c3f20025f806e6a166dff2b4bde3a5237118
Jul 17 01:22:45.692636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:45.692602    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t" event={"ID":"6956ac5e-0202-48c2-9f0c-e5be0e3014c4","Type":"ContainerStarted","Data":"c3e709bda02a12dcb0e25058e0d6c3f20025f806e6a166dff2b4bde3a5237118"}
Jul 17 01:22:46.070072 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:46.070023    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:22:47.702013 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:47.701975    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t" event={"ID":"6956ac5e-0202-48c2-9f0c-e5be0e3014c4","Type":"ContainerStarted","Data":"fdb074af1d9269f6f64c571ecc3e62cf1f6b7e30dc071baf00876d15b62ea09e"}
Jul 17 01:22:47.719552 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:47.719459    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t" podStartSLOduration=1.699242444 podStartE2EDuration="2.719441865s" totalImagesPullingTime="1.020199421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:22:47.717935005 +0000 UTC m=+575.221620880" watchObservedRunningTime="2026-07-17 01:22:47.719441865 +0000 UTC m=+575.223127788"
Jul 17 01:22:55.483818 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:55.483779    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:55.484339 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:55.483849    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:55.496697 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:55.496669    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:22:55.757232 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:22:55.757144    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qnn7t"
Jul 17 01:23:03.553553 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:03.553518    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:23:14.182268 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:14.182241    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:23:14.184635 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:14.183140    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:23:14.189194 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:14.189174    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:23:14.189665 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:14.189647    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:23:24.568308 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.568275    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"]
Jul 17 01:23:24.574237 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.574217    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.576517 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.576483    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 17 01:23:24.581203 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.581177    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"]
Jul 17 01:23:24.686761 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.686727    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.686761 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.686763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46087fb2-c035-48d5-a42a-da522bc1c7a5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.686970 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.686806    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zbtg\" (UniqueName: \"kubernetes.io/projected/46087fb2-c035-48d5-a42a-da522bc1c7a5-kube-api-access-4zbtg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.686970 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.686889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.686970 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.686915    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.686970 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.686947    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.787506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787451    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.787691 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.787691 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.787691 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.787691 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46087fb2-c035-48d5-a42a-da522bc1c7a5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.787918 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787812    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zbtg\" (UniqueName: \"kubernetes.io/projected/46087fb2-c035-48d5-a42a-da522bc1c7a5-kube-api-access-4zbtg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.787982 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787921    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.787982 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.788081 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.787998    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.789981 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.789958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46087fb2-c035-48d5-a42a-da522bc1c7a5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.790553 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.790532    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46087fb2-c035-48d5-a42a-da522bc1c7a5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.795718 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.795694    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zbtg\" (UniqueName: \"kubernetes.io/projected/46087fb2-c035-48d5-a42a-da522bc1c7a5-kube-api-access-4zbtg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl\" (UID: \"46087fb2-c035-48d5-a42a-da522bc1c7a5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:24.885296 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:24.885216    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:25.030874 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:25.030849    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"]
Jul 17 01:23:25.033592 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:23:25.033555    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46087fb2_c035_48d5_a42a_da522bc1c7a5.slice/crio-9696ed1801a9ec02ec61a92bc6b630f02a7d1de9644c88f2d48a6d39ad21db8b WatchSource:0}: Error finding container 9696ed1801a9ec02ec61a92bc6b630f02a7d1de9644c88f2d48a6d39ad21db8b: Status 404 returned error can't find the container with id 9696ed1801a9ec02ec61a92bc6b630f02a7d1de9644c88f2d48a6d39ad21db8b
Jul 17 01:23:25.880479 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:25.880441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl" event={"ID":"46087fb2-c035-48d5-a42a-da522bc1c7a5","Type":"ContainerStarted","Data":"c049b33618a24ad79d48596cc58a9190475e68452be5007c030c8add8318cddf"}
Jul 17 01:23:25.880926 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:25.880516    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl" event={"ID":"46087fb2-c035-48d5-a42a-da522bc1c7a5","Type":"ContainerStarted","Data":"9696ed1801a9ec02ec61a92bc6b630f02a7d1de9644c88f2d48a6d39ad21db8b"}
Jul 17 01:23:25.897042 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:25.896988    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl" podStartSLOduration=1.726699091 podStartE2EDuration="1.896971594s" totalImagesPullingTime="170.272503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:23:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:23:25.89602124 +0000 UTC m=+613.399707125" watchObservedRunningTime="2026-07-17 01:23:25.896971594 +0000 UTC m=+613.400657455"
Jul 17 01:23:26.157950 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:26.157857    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:23:34.886129 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:34.886042    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:34.886129 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:34.886122    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:34.899203 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:34.899177    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:34.934001 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:34.933974    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl"
Jul 17 01:23:35.062328 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:35.062292    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:23:51.762548 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:23:51.762514    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:25:06.863163 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:25:06.863079    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:25:16.355107 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:25:16.355072    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:25:26.863813 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:25:26.863780    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:27:10.364640 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:27:10.364607    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:27:22.465376 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:27:22.465337    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:28:01.060286 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:01.060252    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:28:10.763806 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:10.763724    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:28:14.209566 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:14.209446    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:28:14.215307 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:14.210519    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:28:14.216559 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:14.216538    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:28:14.217219 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:14.217198    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:28:21.070094 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:21.070055    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:28:26.667365 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:26.667335    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:28:37.468098 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:37.468061    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:28:46.968191 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:46.968154    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:28:58.366429 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:28:58.366391    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:29:08.057037 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:29:08.056999    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:29:33.958932 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:29:33.958850    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:29:43.659215 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:29:43.659177    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:30:06.669201 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:30:06.669161    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:30:17.666314 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:30:17.666279    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:30:44.063629 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:30:44.063592    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:30:55.264822 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:30:55.264784    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:31:04.759021 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:31:04.758934    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:31:14.364588 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:31:14.364549    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:33:02.059931 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:02.059899    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:33:12.954408 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:12.954373    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:33:14.238060 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:14.237941    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:33:14.242351 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:14.240616    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:33:14.244947 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:14.244928    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:33:14.247333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:14.247316    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:33:23.864383 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:23.864345    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:33:36.660815 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:36.660767    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:33:48.762372 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:48.762335    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:33:58.158510 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:33:58.158458    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:34:08.267557 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:34:08.267464    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:34:18.256456 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:34:18.256420    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:34:29.359443 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:34:29.359404    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:34:38.156179 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:34:38.156135    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:34:48.764185 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:34:48.764145    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:35:12.961035 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:35:12.961003    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:35:20.171641 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:35:20.171609    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:35:30.367693 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:35:30.367660    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:35:40.163128 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:35:40.163040    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:35:46.660630 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:35:46.660600    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:36:03.067896 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:36:03.067863    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:36:12.659676 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:36:12.659643    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:36:21.661239 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:36:21.661200    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:36:32.564621 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:36:32.564584    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:36:42.165774 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:36:42.165736    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:36:51.853083 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:36:51.853050    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:37:05.355001 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:37:05.354909    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:37:14.370088 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:37:14.370050    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:37:24.263125 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:37:24.263086    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:37:36.455888 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:37:36.455847    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:37:47.065084 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:37:47.065047    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:37:57.261404 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:37:57.261370    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:38:10.460214 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:10.460176    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:38:14.267063 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:14.266939    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:38:14.270949 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:14.269952    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:38:14.274264 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:14.274240    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:38:14.277210 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:14.277190    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:38:21.665822 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:21.665783    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:38:31.459096 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:31.459059    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:38:37.662521 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:37.662415    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:38:50.756226 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:50.756187    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:38:59.364316 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:38:59.364282    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:39:08.175315 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:08.175272    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:39:17.962269 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:17.962231    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:39:19.233353 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.233322    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"]
Jul 17 01:39:19.238101 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.238082    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"
Jul 17 01:39:19.240380 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.240350    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:39:19.241410 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.241375    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:39:19.243995 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.243967    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"]
Jul 17 01:39:19.403085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.403042    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92f91ead-01c7-48fd-9531-8dc3476add47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v\" (UID: \"92f91ead-01c7-48fd-9531-8dc3476add47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"
Jul 17 01:39:19.504470 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.504388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92f91ead-01c7-48fd-9531-8dc3476add47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v\" (UID: \"92f91ead-01c7-48fd-9531-8dc3476add47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"
Jul 17 01:39:19.505122 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.505102    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92f91ead-01c7-48fd-9531-8dc3476add47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v\" (UID: \"92f91ead-01c7-48fd-9531-8dc3476add47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"
Jul 17 01:39:19.550884 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.550848    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"
Jul 17 01:39:19.686452 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.686421    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"]
Jul 17 01:39:19.689893 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:39:19.689867    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f91ead_01c7_48fd_9531_8dc3476add47.slice/crio-3b593b3e73a867ba2891dae6c1220b6ab70908893bfb6f1fa0679fe5398a5338 WatchSource:0}: Error finding container 3b593b3e73a867ba2891dae6c1220b6ab70908893bfb6f1fa0679fe5398a5338: Status 404 returned error can't find the container with id 3b593b3e73a867ba2891dae6c1220b6ab70908893bfb6f1fa0679fe5398a5338
Jul 17 01:39:19.691778 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:19.691758    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 01:39:20.243287 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:20.243254    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" event={"ID":"92f91ead-01c7-48fd-9531-8dc3476add47","Type":"ContainerStarted","Data":"3b593b3e73a867ba2891dae6c1220b6ab70908893bfb6f1fa0679fe5398a5338"}
Jul 17 01:39:36.326925 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:36.326891    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" event={"ID":"92f91ead-01c7-48fd-9531-8dc3476add47","Type":"ContainerStarted","Data":"af14f90d17814dd9f258cd3c6725e24de3110906f2c317615681c5347942e634"}
Jul 17 01:39:36.341562 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:36.341482    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" podStartSLOduration=1.184974234 podStartE2EDuration="17.341465061s" totalImagesPullingTime="16.156490827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:39:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:39:36.339772465 +0000 UTC m=+1583.843458326" watchObservedRunningTime="2026-07-17 01:39:36.341465061 +0000 UTC m=+1583.845150982"
Jul 17 01:39:37.332715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:37.332678    2582 generic.go:356] "Generic (PLEG): container finished" podID="92f91ead-01c7-48fd-9531-8dc3476add47" containerID="af14f90d17814dd9f258cd3c6725e24de3110906f2c317615681c5347942e634" exitCode=7
Jul 17 01:39:37.333172 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:37.332772    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" event={"ID":"92f91ead-01c7-48fd-9531-8dc3476add47","Type":"ContainerDied","Data":"af14f90d17814dd9f258cd3c6725e24de3110906f2c317615681c5347942e634"}
Jul 17 01:39:37.333172 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:37.332922    2582 scope.go:117] "RemoveContainer" containerID="af14f90d17814dd9f258cd3c6725e24de3110906f2c317615681c5347942e634"
Jul 17 01:39:38.339310 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:38.339272    2582 generic.go:356] "Generic (PLEG): container finished" podID="92f91ead-01c7-48fd-9531-8dc3476add47" containerID="6b75fdb6b0e5a05368efb965fcb094a83d709364b39f6ad4e5820629338ed528" exitCode=7
Jul 17 01:39:38.339827 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:38.339330    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" event={"ID":"92f91ead-01c7-48fd-9531-8dc3476add47","Type":"ContainerDied","Data":"6b75fdb6b0e5a05368efb965fcb094a83d709364b39f6ad4e5820629338ed528"}
Jul 17 01:39:38.339827 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:38.339388    2582 scope.go:117] "RemoveContainer" containerID="af14f90d17814dd9f258cd3c6725e24de3110906f2c317615681c5347942e634"
Jul 17 01:39:38.339827 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:38.339641    2582 scope.go:117] "RemoveContainer" containerID="6b75fdb6b0e5a05368efb965fcb094a83d709364b39f6ad4e5820629338ed528"
Jul 17 01:39:38.339978 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:39:38.339920    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v_odh-ai-gateway-infra(92f91ead-01c7-48fd-9531-8dc3476add47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" podUID="92f91ead-01c7-48fd-9531-8dc3476add47"
Jul 17 01:39:39.345676 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:39.345643    2582 scope.go:117] "RemoveContainer" containerID="6b75fdb6b0e5a05368efb965fcb094a83d709364b39f6ad4e5820629338ed528"
Jul 17 01:39:39.346066 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:39:39.345824    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v_odh-ai-gateway-infra(92f91ead-01c7-48fd-9531-8dc3476add47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" podUID="92f91ead-01c7-48fd-9531-8dc3476add47"
Jul 17 01:39:49.984654 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:49.984620    2582 scope.go:117] "RemoveContainer" containerID="6b75fdb6b0e5a05368efb965fcb094a83d709364b39f6ad4e5820629338ed528"
Jul 17 01:39:50.399474 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:50.399436    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" event={"ID":"92f91ead-01c7-48fd-9531-8dc3476add47","Type":"ContainerStarted","Data":"9a450b6b37404e2e8b20e0241e2806de4809efda4a2ab0daa5f09ffd5bcd624d"}
Jul 17 01:39:51.406540 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:51.406484    2582 generic.go:356] "Generic (PLEG): container finished" podID="92f91ead-01c7-48fd-9531-8dc3476add47" containerID="9a450b6b37404e2e8b20e0241e2806de4809efda4a2ab0daa5f09ffd5bcd624d" exitCode=7
Jul 17 01:39:51.406992 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:51.406556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" event={"ID":"92f91ead-01c7-48fd-9531-8dc3476add47","Type":"ContainerDied","Data":"9a450b6b37404e2e8b20e0241e2806de4809efda4a2ab0daa5f09ffd5bcd624d"}
Jul 17 01:39:51.406992 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:51.406618    2582 scope.go:117] "RemoveContainer" containerID="6b75fdb6b0e5a05368efb965fcb094a83d709364b39f6ad4e5820629338ed528"
Jul 17 01:39:51.406992 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:51.406895    2582 scope.go:117] "RemoveContainer" containerID="9a450b6b37404e2e8b20e0241e2806de4809efda4a2ab0daa5f09ffd5bcd624d"
Jul 17 01:39:51.407130 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:39:51.407111    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v_odh-ai-gateway-infra(92f91ead-01c7-48fd-9531-8dc3476add47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" podUID="92f91ead-01c7-48fd-9531-8dc3476add47"
Jul 17 01:39:51.425699 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:51.425668    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"]
Jul 17 01:39:52.549010 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:52.548987    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"
Jul 17 01:39:52.715072 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:52.715031    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92f91ead-01c7-48fd-9531-8dc3476add47-maas-api-service-ca\") pod \"92f91ead-01c7-48fd-9531-8dc3476add47\" (UID: \"92f91ead-01c7-48fd-9531-8dc3476add47\") "
Jul 17 01:39:52.715437 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:52.715412    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92f91ead-01c7-48fd-9531-8dc3476add47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "92f91ead-01c7-48fd-9531-8dc3476add47" (UID: "92f91ead-01c7-48fd-9531-8dc3476add47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:39:52.815935 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:52.815895    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/92f91ead-01c7-48fd-9531-8dc3476add47-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:39:53.419063 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:53.418978    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"
Jul 17 01:39:53.419063 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:53.419015    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v" event={"ID":"92f91ead-01c7-48fd-9531-8dc3476add47","Type":"ContainerDied","Data":"3b593b3e73a867ba2891dae6c1220b6ab70908893bfb6f1fa0679fe5398a5338"}
Jul 17 01:39:53.419286 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:53.419100    2582 scope.go:117] "RemoveContainer" containerID="9a450b6b37404e2e8b20e0241e2806de4809efda4a2ab0daa5f09ffd5bcd624d"
Jul 17 01:39:53.434399 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:53.434373    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"]
Jul 17 01:39:53.439916 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:53.439894    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-7zf7v"]
Jul 17 01:39:54.991288 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:39:54.991252    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" path="/var/lib/kubelet/pods/92f91ead-01c7-48fd-9531-8dc3476add47/volumes"
Jul 17 01:40:29.284799 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.284759    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"]
Jul 17 01:40:29.285309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.285124    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:40:29.285309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.285134    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:40:29.285309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.285144    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:40:29.285309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.285150    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:40:29.285309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.285209    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:40:29.285309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.285217    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:40:29.289558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.289542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"
Jul 17 01:40:29.291856 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.291830    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:40:29.291971 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.291830    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:40:29.295696 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.295654    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"]
Jul 17 01:40:29.418879 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.418846    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/462ce3a0-9da9-4c2d-b27c-d350b532f232-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh\" (UID: \"462ce3a0-9da9-4c2d-b27c-d350b532f232\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"
Jul 17 01:40:29.520161 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.520119    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/462ce3a0-9da9-4c2d-b27c-d350b532f232-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh\" (UID: \"462ce3a0-9da9-4c2d-b27c-d350b532f232\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"
Jul 17 01:40:29.520738 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.520721    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/462ce3a0-9da9-4c2d-b27c-d350b532f232-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh\" (UID: \"462ce3a0-9da9-4c2d-b27c-d350b532f232\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"
Jul 17 01:40:29.600956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.600884    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"
Jul 17 01:40:29.730197 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:29.730165    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"]
Jul 17 01:40:29.733450 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:40:29.733428    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod462ce3a0_9da9_4c2d_b27c_d350b532f232.slice/crio-ab36bd18dd76d3d21bd9990ae69efcc1927e896af6f79e4989b261631093f1de WatchSource:0}: Error finding container ab36bd18dd76d3d21bd9990ae69efcc1927e896af6f79e4989b261631093f1de: Status 404 returned error can't find the container with id ab36bd18dd76d3d21bd9990ae69efcc1927e896af6f79e4989b261631093f1de
Jul 17 01:40:30.599475 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:30.599433    2582 generic.go:356] "Generic (PLEG): container finished" podID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerID="25da7de746095e939192b50fa8c11deb40f1897dcb5bd7f101f0bc2bbe7b6db5" exitCode=7
Jul 17 01:40:30.599870 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:30.599523    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" event={"ID":"462ce3a0-9da9-4c2d-b27c-d350b532f232","Type":"ContainerDied","Data":"25da7de746095e939192b50fa8c11deb40f1897dcb5bd7f101f0bc2bbe7b6db5"}
Jul 17 01:40:30.599870 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:30.599584    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" event={"ID":"462ce3a0-9da9-4c2d-b27c-d350b532f232","Type":"ContainerStarted","Data":"ab36bd18dd76d3d21bd9990ae69efcc1927e896af6f79e4989b261631093f1de"}
Jul 17 01:40:30.599870 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:30.599802    2582 scope.go:117] "RemoveContainer" containerID="25da7de746095e939192b50fa8c11deb40f1897dcb5bd7f101f0bc2bbe7b6db5"
Jul 17 01:40:31.605999 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:31.605911    2582 generic.go:356] "Generic (PLEG): container finished" podID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerID="9807eea713e26a94e66ab8f963aae8678438efe419f611f68e4d792fbd9e5b85" exitCode=7
Jul 17 01:40:31.606379 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:31.605998    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" event={"ID":"462ce3a0-9da9-4c2d-b27c-d350b532f232","Type":"ContainerDied","Data":"9807eea713e26a94e66ab8f963aae8678438efe419f611f68e4d792fbd9e5b85"}
Jul 17 01:40:31.606379 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:31.606055    2582 scope.go:117] "RemoveContainer" containerID="25da7de746095e939192b50fa8c11deb40f1897dcb5bd7f101f0bc2bbe7b6db5"
Jul 17 01:40:31.606379 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:31.606235    2582 scope.go:117] "RemoveContainer" containerID="9807eea713e26a94e66ab8f963aae8678438efe419f611f68e4d792fbd9e5b85"
Jul 17 01:40:31.606515 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:40:31.606468    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh_odh-ai-gateway-infra(462ce3a0-9da9-4c2d-b27c-d350b532f232)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232"
Jul 17 01:40:32.613023 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:32.612995    2582 scope.go:117] "RemoveContainer" containerID="9807eea713e26a94e66ab8f963aae8678438efe419f611f68e4d792fbd9e5b85"
Jul 17 01:40:32.613402 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:40:32.613171    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh_odh-ai-gateway-infra(462ce3a0-9da9-4c2d-b27c-d350b532f232)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232"
Jul 17 01:40:45.985415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:45.985384    2582 scope.go:117] "RemoveContainer" containerID="9807eea713e26a94e66ab8f963aae8678438efe419f611f68e4d792fbd9e5b85"
Jul 17 01:40:46.679177 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:46.679140    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" event={"ID":"462ce3a0-9da9-4c2d-b27c-d350b532f232","Type":"ContainerStarted","Data":"a03196835defce778191f9757c57c886de79d2d2552742b102c6929064851e79"}
Jul 17 01:40:46.695135 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:46.695090    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" podStartSLOduration=17.695074185 podStartE2EDuration="17.695074185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:40:46.693856609 +0000 UTC m=+1654.197542472" watchObservedRunningTime="2026-07-17 01:40:46.695074185 +0000 UTC m=+1654.198760048"
Jul 17 01:40:47.685746 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:47.685712    2582 generic.go:356] "Generic (PLEG): container finished" podID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerID="a03196835defce778191f9757c57c886de79d2d2552742b102c6929064851e79" exitCode=7
Jul 17 01:40:47.686208 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:47.685778    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" event={"ID":"462ce3a0-9da9-4c2d-b27c-d350b532f232","Type":"ContainerDied","Data":"a03196835defce778191f9757c57c886de79d2d2552742b102c6929064851e79"}
Jul 17 01:40:47.686208 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:47.685840    2582 scope.go:117] "RemoveContainer" containerID="9807eea713e26a94e66ab8f963aae8678438efe419f611f68e4d792fbd9e5b85"
Jul 17 01:40:47.686208 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:47.686054    2582 scope.go:117] "RemoveContainer" containerID="a03196835defce778191f9757c57c886de79d2d2552742b102c6929064851e79"
Jul 17 01:40:47.686343 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:40:47.686273    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh_odh-ai-gateway-infra(462ce3a0-9da9-4c2d-b27c-d350b532f232)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232"
Jul 17 01:40:47.710930 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:47.710903    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"]
Jul 17 01:40:48.832339 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:48.832308    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"
Jul 17 01:40:48.993995 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:48.993927    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/462ce3a0-9da9-4c2d-b27c-d350b532f232-maas-api-service-ca\") pod \"462ce3a0-9da9-4c2d-b27c-d350b532f232\" (UID: \"462ce3a0-9da9-4c2d-b27c-d350b532f232\") "
Jul 17 01:40:48.994280 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:48.994253    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/462ce3a0-9da9-4c2d-b27c-d350b532f232-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "462ce3a0-9da9-4c2d-b27c-d350b532f232" (UID: "462ce3a0-9da9-4c2d-b27c-d350b532f232"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:40:49.095068 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:49.095032    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/462ce3a0-9da9-4c2d-b27c-d350b532f232-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:40:49.698712 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:49.698677    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh" event={"ID":"462ce3a0-9da9-4c2d-b27c-d350b532f232","Type":"ContainerDied","Data":"ab36bd18dd76d3d21bd9990ae69efcc1927e896af6f79e4989b261631093f1de"}
Jul 17 01:40:49.698712 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:49.698695    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"
Jul 17 01:40:49.698970 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:49.698746    2582 scope.go:117] "RemoveContainer" containerID="a03196835defce778191f9757c57c886de79d2d2552742b102c6929064851e79"
Jul 17 01:40:49.724201 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:49.724178    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"]
Jul 17 01:40:49.729678 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:49.729657    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2drhh"]
Jul 17 01:40:50.990588 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:40:50.990554    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" path="/var/lib/kubelet/pods/462ce3a0-9da9-4c2d-b27c-d350b532f232/volumes"
Jul 17 01:41:29.327020 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.326980    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"]
Jul 17 01:41:29.327432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327328    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:41:29.327432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327338    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:41:29.327432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327350    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327355    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327364    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327370    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327380    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327385    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327697 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327443    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327697 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327450    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327697 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327457    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="462ce3a0-9da9-4c2d-b27c-d350b532f232" containerName="revoke-keys"
Jul 17 01:41:29.327697 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.327462    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="92f91ead-01c7-48fd-9531-8dc3476add47" containerName="revoke-keys"
Jul 17 01:41:29.331406 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.331388    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"
Jul 17 01:41:29.333597 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.333579    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:41:29.333849 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.333819    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:41:29.337244 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.337214    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"]
Jul 17 01:41:29.444172 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.444136    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dbf36b5-e021-47fa-9a25-ccae7b77657d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6\" (UID: \"0dbf36b5-e021-47fa-9a25-ccae7b77657d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"
Jul 17 01:41:29.544832 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.544791    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dbf36b5-e021-47fa-9a25-ccae7b77657d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6\" (UID: \"0dbf36b5-e021-47fa-9a25-ccae7b77657d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"
Jul 17 01:41:29.545405 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.545384    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dbf36b5-e021-47fa-9a25-ccae7b77657d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6\" (UID: \"0dbf36b5-e021-47fa-9a25-ccae7b77657d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"
Jul 17 01:41:29.644049 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.643973    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"
Jul 17 01:41:29.775307 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.775284    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"]
Jul 17 01:41:29.777706 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:41:29.777682    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dbf36b5_e021_47fa_9a25_ccae7b77657d.slice/crio-c5d4fb7c524d598595a097a55138f315abe87e6f06980353606949303750e2d7 WatchSource:0}: Error finding container c5d4fb7c524d598595a097a55138f315abe87e6f06980353606949303750e2d7: Status 404 returned error can't find the container with id c5d4fb7c524d598595a097a55138f315abe87e6f06980353606949303750e2d7
Jul 17 01:41:29.889857 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:29.889822    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6" event={"ID":"0dbf36b5-e021-47fa-9a25-ccae7b77657d","Type":"ContainerStarted","Data":"c5d4fb7c524d598595a097a55138f315abe87e6f06980353606949303750e2d7"}
Jul 17 01:41:30.896177 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:30.896144    2582 generic.go:356] "Generic (PLEG): container finished" podID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerID="6fdcbe58144e27acf808bb5a2f1825f4f51a14cad9e92fcdf9183f9043ea9476" exitCode=7
Jul 17 01:41:30.896654 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:30.896232    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6" event={"ID":"0dbf36b5-e021-47fa-9a25-ccae7b77657d","Type":"ContainerDied","Data":"6fdcbe58144e27acf808bb5a2f1825f4f51a14cad9e92fcdf9183f9043ea9476"}
Jul 17 01:41:30.896654 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:30.896408    2582 scope.go:117] "RemoveContainer" containerID="6fdcbe58144e27acf808bb5a2f1825f4f51a14cad9e92fcdf9183f9043ea9476"
Jul 17 01:41:31.903243 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:31.903148    2582 generic.go:356] "Generic (PLEG): container finished" podID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerID="b7ca8e81705a815cd09ff560b1662efca848b7b9914e5cf260201d52b233b716" exitCode=7
Jul 17 01:41:31.903688 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:31.903248    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6" event={"ID":"0dbf36b5-e021-47fa-9a25-ccae7b77657d","Type":"ContainerDied","Data":"b7ca8e81705a815cd09ff560b1662efca848b7b9914e5cf260201d52b233b716"}
Jul 17 01:41:31.903688 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:31.903316    2582 scope.go:117] "RemoveContainer" containerID="6fdcbe58144e27acf808bb5a2f1825f4f51a14cad9e92fcdf9183f9043ea9476"
Jul 17 01:41:31.903688 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:31.903441    2582 scope.go:117] "RemoveContainer" containerID="b7ca8e81705a815cd09ff560b1662efca848b7b9914e5cf260201d52b233b716"
Jul 17 01:41:31.903688 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:41:31.903668    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6_odh-ai-gateway-infra(0dbf36b5-e021-47fa-9a25-ccae7b77657d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d"
Jul 17 01:41:32.909632 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:32.909601    2582 scope.go:117] "RemoveContainer" containerID="b7ca8e81705a815cd09ff560b1662efca848b7b9914e5cf260201d52b233b716"
Jul 17 01:41:32.909987 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:41:32.909803    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6_odh-ai-gateway-infra(0dbf36b5-e021-47fa-9a25-ccae7b77657d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d"
Jul 17 01:41:45.985558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:45.985467    2582 scope.go:117] "RemoveContainer" containerID="b7ca8e81705a815cd09ff560b1662efca848b7b9914e5cf260201d52b233b716"
Jul 17 01:41:46.974367 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:46.974333    2582 generic.go:356] "Generic (PLEG): container finished" podID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerID="22602da91dbdf302b3603dde297446bfba1b03121034daabe5c407a9873fc4cf" exitCode=7
Jul 17 01:41:46.974602 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:46.974402    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6" event={"ID":"0dbf36b5-e021-47fa-9a25-ccae7b77657d","Type":"ContainerDied","Data":"22602da91dbdf302b3603dde297446bfba1b03121034daabe5c407a9873fc4cf"}
Jul 17 01:41:46.974602 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:46.974475    2582 scope.go:117] "RemoveContainer" containerID="b7ca8e81705a815cd09ff560b1662efca848b7b9914e5cf260201d52b233b716"
Jul 17 01:41:46.974814 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:46.974799    2582 scope.go:117] "RemoveContainer" containerID="22602da91dbdf302b3603dde297446bfba1b03121034daabe5c407a9873fc4cf"
Jul 17 01:41:46.975006 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:41:46.974990    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6_odh-ai-gateway-infra(0dbf36b5-e021-47fa-9a25-ccae7b77657d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d"
Jul 17 01:41:47.998421 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:47.998386    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"]
Jul 17 01:41:48.137885 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:48.137863    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"
Jul 17 01:41:48.211737 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:48.211702    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dbf36b5-e021-47fa-9a25-ccae7b77657d-maas-api-service-ca\") pod \"0dbf36b5-e021-47fa-9a25-ccae7b77657d\" (UID: \"0dbf36b5-e021-47fa-9a25-ccae7b77657d\") "
Jul 17 01:41:48.212092 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:48.212070    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dbf36b5-e021-47fa-9a25-ccae7b77657d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0dbf36b5-e021-47fa-9a25-ccae7b77657d" (UID: "0dbf36b5-e021-47fa-9a25-ccae7b77657d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:41:48.313008 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:48.312919    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dbf36b5-e021-47fa-9a25-ccae7b77657d-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:41:48.987344 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:48.987313    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"
Jul 17 01:41:48.989998 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:48.989970    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6" event={"ID":"0dbf36b5-e021-47fa-9a25-ccae7b77657d","Type":"ContainerDied","Data":"c5d4fb7c524d598595a097a55138f315abe87e6f06980353606949303750e2d7"}
Jul 17 01:41:48.990122 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:48.990022    2582 scope.go:117] "RemoveContainer" containerID="22602da91dbdf302b3603dde297446bfba1b03121034daabe5c407a9873fc4cf"
Jul 17 01:41:49.009780 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:49.009756    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"]
Jul 17 01:41:49.014994 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:49.014972    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-zfkt6"]
Jul 17 01:41:50.990958 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:41:50.990925    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" path="/var/lib/kubelet/pods/0dbf36b5-e021-47fa-9a25-ccae7b77657d/volumes"
Jul 17 01:42:19.369994 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.369957    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"]
Jul 17 01:42:19.370441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.370299    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:42:19.370441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.370310    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:42:19.370441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.370328    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:42:19.370441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.370334    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:42:19.370441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.370391    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:42:19.370441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.370401    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:42:19.370441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.370408    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:42:19.374640 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.374624    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"
Jul 17 01:42:19.377022 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.377002    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:42:19.377115 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.377069    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:42:19.380370 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.380347    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"]
Jul 17 01:42:19.494967 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.494931    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73af4881-e0a2-46d1-aa38-d2a61e023429-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c\" (UID: \"73af4881-e0a2-46d1-aa38-d2a61e023429\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"
Jul 17 01:42:19.596505 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.596452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73af4881-e0a2-46d1-aa38-d2a61e023429-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c\" (UID: \"73af4881-e0a2-46d1-aa38-d2a61e023429\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"
Jul 17 01:42:19.597077 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.597057    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73af4881-e0a2-46d1-aa38-d2a61e023429-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c\" (UID: \"73af4881-e0a2-46d1-aa38-d2a61e023429\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"
Jul 17 01:42:19.687996 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.687970    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"
Jul 17 01:42:19.821315 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:19.821287    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"]
Jul 17 01:42:19.823023 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:42:19.822993    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73af4881_e0a2_46d1_aa38_d2a61e023429.slice/crio-6f7ab217056794402bbdd73b076b2d5fabf2470192ffcb00d0f3d3838b3e6137 WatchSource:0}: Error finding container 6f7ab217056794402bbdd73b076b2d5fabf2470192ffcb00d0f3d3838b3e6137: Status 404 returned error can't find the container with id 6f7ab217056794402bbdd73b076b2d5fabf2470192ffcb00d0f3d3838b3e6137
Jul 17 01:42:20.128298 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:20.128267    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" event={"ID":"73af4881-e0a2-46d1-aa38-d2a61e023429","Type":"ContainerStarted","Data":"ed6b83706d99168a3c67a65bf967b074e3b0bed1bffcd5292a923d9737a6e92d"}
Jul 17 01:42:20.128412 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:20.128323    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" event={"ID":"73af4881-e0a2-46d1-aa38-d2a61e023429","Type":"ContainerStarted","Data":"6f7ab217056794402bbdd73b076b2d5fabf2470192ffcb00d0f3d3838b3e6137"}
Jul 17 01:42:20.143681 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:20.143629    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" podStartSLOduration=1.1436131 podStartE2EDuration="1.1436131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:42:20.142559959 +0000 UTC m=+1747.646245820" watchObservedRunningTime="2026-07-17 01:42:20.1436131 +0000 UTC m=+1747.647298962"
Jul 17 01:42:21.135027 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:21.134995    2582 generic.go:356] "Generic (PLEG): container finished" podID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerID="ed6b83706d99168a3c67a65bf967b074e3b0bed1bffcd5292a923d9737a6e92d" exitCode=7
Jul 17 01:42:21.135437 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:21.135035    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" event={"ID":"73af4881-e0a2-46d1-aa38-d2a61e023429","Type":"ContainerDied","Data":"ed6b83706d99168a3c67a65bf967b074e3b0bed1bffcd5292a923d9737a6e92d"}
Jul 17 01:42:21.135437 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:21.135240    2582 scope.go:117] "RemoveContainer" containerID="ed6b83706d99168a3c67a65bf967b074e3b0bed1bffcd5292a923d9737a6e92d"
Jul 17 01:42:21.976677 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:21.976638    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:42:22.144425 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:22.144343    2582 generic.go:356] "Generic (PLEG): container finished" podID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerID="deba8c11594a557265f90b3a18beb9bee9f8c999124786b4e63c518e92bbc4aa" exitCode=7
Jul 17 01:42:22.144885 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:22.144432    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" event={"ID":"73af4881-e0a2-46d1-aa38-d2a61e023429","Type":"ContainerDied","Data":"deba8c11594a557265f90b3a18beb9bee9f8c999124786b4e63c518e92bbc4aa"}
Jul 17 01:42:22.144885 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:22.144527    2582 scope.go:117] "RemoveContainer" containerID="ed6b83706d99168a3c67a65bf967b074e3b0bed1bffcd5292a923d9737a6e92d"
Jul 17 01:42:22.144885 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:22.144635    2582 scope.go:117] "RemoveContainer" containerID="deba8c11594a557265f90b3a18beb9bee9f8c999124786b4e63c518e92bbc4aa"
Jul 17 01:42:22.144885 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:42:22.144835    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c_odh-ai-gateway-infra(73af4881-e0a2-46d1-aa38-d2a61e023429)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429"
Jul 17 01:42:23.151102 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:23.151070    2582 scope.go:117] "RemoveContainer" containerID="deba8c11594a557265f90b3a18beb9bee9f8c999124786b4e63c518e92bbc4aa"
Jul 17 01:42:23.151471 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:42:23.151268    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c_odh-ai-gateway-infra(73af4881-e0a2-46d1-aa38-d2a61e023429)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429"
Jul 17 01:42:35.460703 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:35.460665    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:42:36.985323 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:36.985291    2582 scope.go:117] "RemoveContainer" containerID="deba8c11594a557265f90b3a18beb9bee9f8c999124786b4e63c518e92bbc4aa"
Jul 17 01:42:38.221866 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:38.221835    2582 generic.go:356] "Generic (PLEG): container finished" podID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerID="d4c71b1723cf4be96377f636728753a6a36c5675a9d6671e5bff7619b87ec0d9" exitCode=6
Jul 17 01:42:38.222281 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:38.221909    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" event={"ID":"73af4881-e0a2-46d1-aa38-d2a61e023429","Type":"ContainerDied","Data":"d4c71b1723cf4be96377f636728753a6a36c5675a9d6671e5bff7619b87ec0d9"}
Jul 17 01:42:38.222281 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:38.221971    2582 scope.go:117] "RemoveContainer" containerID="deba8c11594a557265f90b3a18beb9bee9f8c999124786b4e63c518e92bbc4aa"
Jul 17 01:42:38.222281 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:38.222234    2582 scope.go:117] "RemoveContainer" containerID="d4c71b1723cf4be96377f636728753a6a36c5675a9d6671e5bff7619b87ec0d9"
Jul 17 01:42:38.222516 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:42:38.222464    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c_odh-ai-gateway-infra(73af4881-e0a2-46d1-aa38-d2a61e023429)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429"
Jul 17 01:42:39.247042 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:39.247005    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"]
Jul 17 01:42:39.387129 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:39.387107    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"
Jul 17 01:42:39.579735 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:39.579632    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73af4881-e0a2-46d1-aa38-d2a61e023429-maas-api-service-ca\") pod \"73af4881-e0a2-46d1-aa38-d2a61e023429\" (UID: \"73af4881-e0a2-46d1-aa38-d2a61e023429\") "
Jul 17 01:42:39.580002 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:39.579980    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73af4881-e0a2-46d1-aa38-d2a61e023429-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73af4881-e0a2-46d1-aa38-d2a61e023429" (UID: "73af4881-e0a2-46d1-aa38-d2a61e023429"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:42:39.681116 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:39.681080    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73af4881-e0a2-46d1-aa38-d2a61e023429-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:42:40.233896 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:40.233860    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c" event={"ID":"73af4881-e0a2-46d1-aa38-d2a61e023429","Type":"ContainerDied","Data":"6f7ab217056794402bbdd73b076b2d5fabf2470192ffcb00d0f3d3838b3e6137"}
Jul 17 01:42:40.233896 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:40.233887    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"
Jul 17 01:42:40.234136 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:40.233936    2582 scope.go:117] "RemoveContainer" containerID="d4c71b1723cf4be96377f636728753a6a36c5675a9d6671e5bff7619b87ec0d9"
Jul 17 01:42:40.254849 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:40.254822    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"]
Jul 17 01:42:40.260007 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:40.259981    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-d6p9c"]
Jul 17 01:42:40.996709 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:42:40.996674    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" path="/var/lib/kubelet/pods/73af4881-e0a2-46d1-aa38-d2a61e023429/volumes"
Jul 17 01:43:14.297440 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:14.297321    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:43:14.305039 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:14.299891    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:43:14.305039 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:14.304078    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:43:14.306849 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:14.306832    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:43:19.406396 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406360    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"]
Jul 17 01:43:19.406803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406718    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:43:19.406803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406729    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dbf36b5-e021-47fa-9a25-ccae7b77657d" containerName="revoke-keys"
Jul 17 01:43:19.406803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406743    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.406803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406748    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.406803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406763    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.406803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406769    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.406803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406775    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.406803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406780    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.407069 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406835    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.407069 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406843    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.407069 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.406849    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="73af4881-e0a2-46d1-aa38-d2a61e023429" containerName="revoke-keys"
Jul 17 01:43:19.409678 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.409655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"
Jul 17 01:43:19.412044 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.412016    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:43:19.412264 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.412243    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:43:19.416612 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.416582    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"]
Jul 17 01:43:19.539311 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.539269    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16f5b2fd-4a8d-4805-8471-5440cdb890ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7\" (UID: \"16f5b2fd-4a8d-4805-8471-5440cdb890ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"
Jul 17 01:43:19.640478 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.640440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16f5b2fd-4a8d-4805-8471-5440cdb890ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7\" (UID: \"16f5b2fd-4a8d-4805-8471-5440cdb890ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"
Jul 17 01:43:19.641076 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.641056    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16f5b2fd-4a8d-4805-8471-5440cdb890ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7\" (UID: \"16f5b2fd-4a8d-4805-8471-5440cdb890ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"
Jul 17 01:43:19.721694 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.721649    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"
Jul 17 01:43:19.856584 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:19.856555    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"]
Jul 17 01:43:19.858696 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:43:19.858667    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f5b2fd_4a8d_4805_8471_5440cdb890ba.slice/crio-933e0cbaec161593b05da174a28b7c5db007acb9da54ad098fb4ab4fc8726d52 WatchSource:0}: Error finding container 933e0cbaec161593b05da174a28b7c5db007acb9da54ad098fb4ab4fc8726d52: Status 404 returned error can't find the container with id 933e0cbaec161593b05da174a28b7c5db007acb9da54ad098fb4ab4fc8726d52
Jul 17 01:43:20.417085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:20.416990    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" event={"ID":"16f5b2fd-4a8d-4805-8471-5440cdb890ba","Type":"ContainerStarted","Data":"132d116f24fffbfeceb7694a177caec395f079278b5409cffee49ac65a6f4520"}
Jul 17 01:43:20.417085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:20.417059    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" event={"ID":"16f5b2fd-4a8d-4805-8471-5440cdb890ba","Type":"ContainerStarted","Data":"933e0cbaec161593b05da174a28b7c5db007acb9da54ad098fb4ab4fc8726d52"}
Jul 17 01:43:20.431717 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:20.431666    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" podStartSLOduration=1.431650075 podStartE2EDuration="1.431650075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:43:20.430777341 +0000 UTC m=+1807.934463204" watchObservedRunningTime="2026-07-17 01:43:20.431650075 +0000 UTC m=+1807.935335936"
Jul 17 01:43:21.422745 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:21.422709    2582 generic.go:356] "Generic (PLEG): container finished" podID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerID="132d116f24fffbfeceb7694a177caec395f079278b5409cffee49ac65a6f4520" exitCode=6
Jul 17 01:43:21.423114 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:21.422775    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" event={"ID":"16f5b2fd-4a8d-4805-8471-5440cdb890ba","Type":"ContainerDied","Data":"132d116f24fffbfeceb7694a177caec395f079278b5409cffee49ac65a6f4520"}
Jul 17 01:43:21.423114 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:21.422935    2582 scope.go:117] "RemoveContainer" containerID="132d116f24fffbfeceb7694a177caec395f079278b5409cffee49ac65a6f4520"
Jul 17 01:43:22.430084 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:22.430044    2582 generic.go:356] "Generic (PLEG): container finished" podID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerID="f1fa574e9eb3bc151c0f18d9599a33aa5e7a96424485c278742202b6c74fdbd7" exitCode=6
Jul 17 01:43:22.430506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:22.430136    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" event={"ID":"16f5b2fd-4a8d-4805-8471-5440cdb890ba","Type":"ContainerDied","Data":"f1fa574e9eb3bc151c0f18d9599a33aa5e7a96424485c278742202b6c74fdbd7"}
Jul 17 01:43:22.430506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:22.430228    2582 scope.go:117] "RemoveContainer" containerID="132d116f24fffbfeceb7694a177caec395f079278b5409cffee49ac65a6f4520"
Jul 17 01:43:22.430506 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:22.430438    2582 scope.go:117] "RemoveContainer" containerID="f1fa574e9eb3bc151c0f18d9599a33aa5e7a96424485c278742202b6c74fdbd7"
Jul 17 01:43:22.430695 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:43:22.430671    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7_odh-ai-gateway-infra(16f5b2fd-4a8d-4805-8471-5440cdb890ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba"
Jul 17 01:43:23.436439 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:23.436413    2582 scope.go:117] "RemoveContainer" containerID="f1fa574e9eb3bc151c0f18d9599a33aa5e7a96424485c278742202b6c74fdbd7"
Jul 17 01:43:23.436851 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:43:23.436632    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7_odh-ai-gateway-infra(16f5b2fd-4a8d-4805-8471-5440cdb890ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba"
Jul 17 01:43:36.985250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:36.985215    2582 scope.go:117] "RemoveContainer" containerID="f1fa574e9eb3bc151c0f18d9599a33aa5e7a96424485c278742202b6c74fdbd7"
Jul 17 01:43:37.503498 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:37.503466    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" event={"ID":"16f5b2fd-4a8d-4805-8471-5440cdb890ba","Type":"ContainerStarted","Data":"c2018d9d6d31194f5d12883ee2e1da0f8373a13657a2f770b1bf102e603d9c41"}
Jul 17 01:43:38.510086 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:38.510053    2582 generic.go:356] "Generic (PLEG): container finished" podID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerID="c2018d9d6d31194f5d12883ee2e1da0f8373a13657a2f770b1bf102e603d9c41" exitCode=6
Jul 17 01:43:38.510511 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:38.510123    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" event={"ID":"16f5b2fd-4a8d-4805-8471-5440cdb890ba","Type":"ContainerDied","Data":"c2018d9d6d31194f5d12883ee2e1da0f8373a13657a2f770b1bf102e603d9c41"}
Jul 17 01:43:38.510511 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:38.510199    2582 scope.go:117] "RemoveContainer" containerID="f1fa574e9eb3bc151c0f18d9599a33aa5e7a96424485c278742202b6c74fdbd7"
Jul 17 01:43:38.510511 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:38.510463    2582 scope.go:117] "RemoveContainer" containerID="c2018d9d6d31194f5d12883ee2e1da0f8373a13657a2f770b1bf102e603d9c41"
Jul 17 01:43:38.510701 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:43:38.510681    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7_odh-ai-gateway-infra(16f5b2fd-4a8d-4805-8471-5440cdb890ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba"
Jul 17 01:43:38.530990 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:38.530964    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"]
Jul 17 01:43:39.653090 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:39.653060    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"
Jul 17 01:43:39.721535 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:39.721473    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16f5b2fd-4a8d-4805-8471-5440cdb890ba-maas-api-service-ca\") pod \"16f5b2fd-4a8d-4805-8471-5440cdb890ba\" (UID: \"16f5b2fd-4a8d-4805-8471-5440cdb890ba\") "
Jul 17 01:43:39.721882 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:39.721855    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16f5b2fd-4a8d-4805-8471-5440cdb890ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "16f5b2fd-4a8d-4805-8471-5440cdb890ba" (UID: "16f5b2fd-4a8d-4805-8471-5440cdb890ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:43:39.822894 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:39.822807    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16f5b2fd-4a8d-4805-8471-5440cdb890ba-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:43:40.257953 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:40.257920    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:43:40.522340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:40.522258    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"
Jul 17 01:43:40.522340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:40.522287    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7" event={"ID":"16f5b2fd-4a8d-4805-8471-5440cdb890ba","Type":"ContainerDied","Data":"933e0cbaec161593b05da174a28b7c5db007acb9da54ad098fb4ab4fc8726d52"}
Jul 17 01:43:40.522578 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:40.522349    2582 scope.go:117] "RemoveContainer" containerID="c2018d9d6d31194f5d12883ee2e1da0f8373a13657a2f770b1bf102e603d9c41"
Jul 17 01:43:40.548820 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:40.548785    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"]
Jul 17 01:43:40.550522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:40.550476    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-42rc7"]
Jul 17 01:43:40.990048 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:40.990014    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" path="/var/lib/kubelet/pods/16f5b2fd-4a8d-4805-8471-5440cdb890ba/volumes"
Jul 17 01:43:55.499254 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.499222    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"]
Jul 17 01:43:55.499781 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.499760    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:43:55.499826 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.499786    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:43:55.499826 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.499816    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:43:55.499891 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.499825    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:43:55.499963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.499952    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:43:55.499997 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.499969    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:43:55.499997 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.499978    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:43:55.502804 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.502789    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"
Jul 17 01:43:55.505340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.505314    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:43:55.505340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.505327    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:43:55.509763 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.509741    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"]
Jul 17 01:43:55.669108 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.669071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2694aa3c-90a9-4b06-a91a-0f89a02c873f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv\" (UID: \"2694aa3c-90a9-4b06-a91a-0f89a02c873f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"
Jul 17 01:43:55.769860 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.769777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2694aa3c-90a9-4b06-a91a-0f89a02c873f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv\" (UID: \"2694aa3c-90a9-4b06-a91a-0f89a02c873f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"
Jul 17 01:43:55.770398 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.770377    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2694aa3c-90a9-4b06-a91a-0f89a02c873f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv\" (UID: \"2694aa3c-90a9-4b06-a91a-0f89a02c873f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"
Jul 17 01:43:55.815764 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.815732    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"
Jul 17 01:43:55.949857 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:55.949825    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"]
Jul 17 01:43:55.952470 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:43:55.952442    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2694aa3c_90a9_4b06_a91a_0f89a02c873f.slice/crio-6ac7968391beeee73b32fdd8845a28f7ebd9c9a52d04340c23c566fd8adeca60 WatchSource:0}: Error finding container 6ac7968391beeee73b32fdd8845a28f7ebd9c9a52d04340c23c566fd8adeca60: Status 404 returned error can't find the container with id 6ac7968391beeee73b32fdd8845a28f7ebd9c9a52d04340c23c566fd8adeca60
Jul 17 01:43:56.598266 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:56.598231    2582 generic.go:356] "Generic (PLEG): container finished" podID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerID="d4dfc0858bd8831616f47fcdbe5537548ea515830fef4b946baebe9e14d3094f" exitCode=7
Jul 17 01:43:56.598721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:56.598279    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv" event={"ID":"2694aa3c-90a9-4b06-a91a-0f89a02c873f","Type":"ContainerDied","Data":"d4dfc0858bd8831616f47fcdbe5537548ea515830fef4b946baebe9e14d3094f"}
Jul 17 01:43:56.598721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:56.598337    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv" event={"ID":"2694aa3c-90a9-4b06-a91a-0f89a02c873f","Type":"ContainerStarted","Data":"6ac7968391beeee73b32fdd8845a28f7ebd9c9a52d04340c23c566fd8adeca60"}
Jul 17 01:43:56.598721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:56.598540    2582 scope.go:117] "RemoveContainer" containerID="d4dfc0858bd8831616f47fcdbe5537548ea515830fef4b946baebe9e14d3094f"
Jul 17 01:43:57.160032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:57.159952    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:43:57.605031 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:57.604992    2582 generic.go:356] "Generic (PLEG): container finished" podID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerID="5928ffa8880c10605d812d1cf3b1acfc1a6ca69d12e34dfbce86f5168cfa6cfe" exitCode=7
Jul 17 01:43:57.605435 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:57.605086    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv" event={"ID":"2694aa3c-90a9-4b06-a91a-0f89a02c873f","Type":"ContainerDied","Data":"5928ffa8880c10605d812d1cf3b1acfc1a6ca69d12e34dfbce86f5168cfa6cfe"}
Jul 17 01:43:57.605435 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:57.605147    2582 scope.go:117] "RemoveContainer" containerID="d4dfc0858bd8831616f47fcdbe5537548ea515830fef4b946baebe9e14d3094f"
Jul 17 01:43:57.605435 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:57.605288    2582 scope.go:117] "RemoveContainer" containerID="5928ffa8880c10605d812d1cf3b1acfc1a6ca69d12e34dfbce86f5168cfa6cfe"
Jul 17 01:43:57.605583 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:43:57.605544    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv_odh-ai-gateway-infra(2694aa3c-90a9-4b06-a91a-0f89a02c873f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f"
Jul 17 01:43:58.612386 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:43:58.612354    2582 scope.go:117] "RemoveContainer" containerID="5928ffa8880c10605d812d1cf3b1acfc1a6ca69d12e34dfbce86f5168cfa6cfe"
Jul 17 01:43:58.612889 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:43:58.612585    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv_odh-ai-gateway-infra(2694aa3c-90a9-4b06-a91a-0f89a02c873f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f"
Jul 17 01:44:10.984960 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:10.984928    2582 scope.go:117] "RemoveContainer" containerID="5928ffa8880c10605d812d1cf3b1acfc1a6ca69d12e34dfbce86f5168cfa6cfe"
Jul 17 01:44:11.674116 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:11.674085    2582 generic.go:356] "Generic (PLEG): container finished" podID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerID="1de88538fcd79714ecdfd76b36d3a079f785511cd4b030447f5f7f2a533a9fa9" exitCode=7
Jul 17 01:44:11.674277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:11.674152    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv" event={"ID":"2694aa3c-90a9-4b06-a91a-0f89a02c873f","Type":"ContainerDied","Data":"1de88538fcd79714ecdfd76b36d3a079f785511cd4b030447f5f7f2a533a9fa9"}
Jul 17 01:44:11.674277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:11.674214    2582 scope.go:117] "RemoveContainer" containerID="5928ffa8880c10605d812d1cf3b1acfc1a6ca69d12e34dfbce86f5168cfa6cfe"
Jul 17 01:44:11.674603 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:11.674586    2582 scope.go:117] "RemoveContainer" containerID="1de88538fcd79714ecdfd76b36d3a079f785511cd4b030447f5f7f2a533a9fa9"
Jul 17 01:44:11.674854 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:11.674830    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv_odh-ai-gateway-infra(2694aa3c-90a9-4b06-a91a-0f89a02c873f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f"
Jul 17 01:44:12.697081 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:12.697051    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"]
Jul 17 01:44:12.860507 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:12.860464    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"
Jul 17 01:44:12.936250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:12.936218    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2694aa3c-90a9-4b06-a91a-0f89a02c873f-maas-api-service-ca\") pod \"2694aa3c-90a9-4b06-a91a-0f89a02c873f\" (UID: \"2694aa3c-90a9-4b06-a91a-0f89a02c873f\") "
Jul 17 01:44:12.936641 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:12.936617    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2694aa3c-90a9-4b06-a91a-0f89a02c873f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2694aa3c-90a9-4b06-a91a-0f89a02c873f" (UID: "2694aa3c-90a9-4b06-a91a-0f89a02c873f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:44:13.037605 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:13.037574    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2694aa3c-90a9-4b06-a91a-0f89a02c873f-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:44:13.693168 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:13.693131    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv" event={"ID":"2694aa3c-90a9-4b06-a91a-0f89a02c873f","Type":"ContainerDied","Data":"6ac7968391beeee73b32fdd8845a28f7ebd9c9a52d04340c23c566fd8adeca60"}
Jul 17 01:44:13.693370 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:13.693191    2582 scope.go:117] "RemoveContainer" containerID="1de88538fcd79714ecdfd76b36d3a079f785511cd4b030447f5f7f2a533a9fa9"
Jul 17 01:44:13.693370 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:13.693140    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"
Jul 17 01:44:13.708340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:13.708315    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"]
Jul 17 01:44:13.712245 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:13.712222    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-44lwv"]
Jul 17 01:44:14.990808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:14.990772    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" path="/var/lib/kubelet/pods/2694aa3c-90a9-4b06-a91a-0f89a02c873f/volumes"
Jul 17 01:44:19.447430 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447395    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"]
Jul 17 01:44:19.447858 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447767    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:19.447858 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447779    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:19.447858 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447796    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:44:19.447858 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447802    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="16f5b2fd-4a8d-4805-8471-5440cdb890ba" containerName="revoke-keys"
Jul 17 01:44:19.447858 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447812    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:19.447858 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447818    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:19.448046 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447870    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:19.448046 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.447879    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:19.450985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.450969    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"
Jul 17 01:44:19.453235 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.453209    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:44:19.453398 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.453264    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:44:19.457454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.457423    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"]
Jul 17 01:44:19.496350 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.496318    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/159180d3-e207-4764-9671-247894a01f3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4\" (UID: \"159180d3-e207-4764-9671-247894a01f3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"
Jul 17 01:44:19.596844 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.596809    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/159180d3-e207-4764-9671-247894a01f3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4\" (UID: \"159180d3-e207-4764-9671-247894a01f3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"
Jul 17 01:44:19.597454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.597432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/159180d3-e207-4764-9671-247894a01f3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4\" (UID: \"159180d3-e207-4764-9671-247894a01f3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"
Jul 17 01:44:19.762954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.762853    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"
Jul 17 01:44:19.898139 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:19.898110    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"]
Jul 17 01:44:19.901702 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:44:19.901671    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159180d3_e207_4764_9671_247894a01f3f.slice/crio-b24739607e62d38354f7867c648219a93a344d1c610e77513e4322731fde8600 WatchSource:0}: Error finding container b24739607e62d38354f7867c648219a93a344d1c610e77513e4322731fde8600: Status 404 returned error can't find the container with id b24739607e62d38354f7867c648219a93a344d1c610e77513e4322731fde8600
Jul 17 01:44:20.728556 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:20.728522    2582 generic.go:356] "Generic (PLEG): container finished" podID="159180d3-e207-4764-9671-247894a01f3f" containerID="8ae39ae976de1f3db52508c5e2fd737fb4a2ffc47abbc68b142bb574b7c832b9" exitCode=6
Jul 17 01:44:20.729132 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:20.728595    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4" event={"ID":"159180d3-e207-4764-9671-247894a01f3f","Type":"ContainerDied","Data":"8ae39ae976de1f3db52508c5e2fd737fb4a2ffc47abbc68b142bb574b7c832b9"}
Jul 17 01:44:20.729132 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:20.728634    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4" event={"ID":"159180d3-e207-4764-9671-247894a01f3f","Type":"ContainerStarted","Data":"b24739607e62d38354f7867c648219a93a344d1c610e77513e4322731fde8600"}
Jul 17 01:44:20.729132 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:20.728881    2582 scope.go:117] "RemoveContainer" containerID="8ae39ae976de1f3db52508c5e2fd737fb4a2ffc47abbc68b142bb574b7c832b9"
Jul 17 01:44:21.735588 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:21.735553    2582 generic.go:356] "Generic (PLEG): container finished" podID="159180d3-e207-4764-9671-247894a01f3f" containerID="f993f26245235af2232b834d9bbafed60f19f17df735ff647dd32ebf544d6e44" exitCode=6
Jul 17 01:44:21.735994 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:21.735637    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4" event={"ID":"159180d3-e207-4764-9671-247894a01f3f","Type":"ContainerDied","Data":"f993f26245235af2232b834d9bbafed60f19f17df735ff647dd32ebf544d6e44"}
Jul 17 01:44:21.735994 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:21.735694    2582 scope.go:117] "RemoveContainer" containerID="8ae39ae976de1f3db52508c5e2fd737fb4a2ffc47abbc68b142bb574b7c832b9"
Jul 17 01:44:21.735994 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:21.735902    2582 scope.go:117] "RemoveContainer" containerID="f993f26245235af2232b834d9bbafed60f19f17df735ff647dd32ebf544d6e44"
Jul 17 01:44:21.736143 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:21.736122    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4_odh-ai-gateway-infra(159180d3-e207-4764-9671-247894a01f3f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4" podUID="159180d3-e207-4764-9671-247894a01f3f"
Jul 17 01:44:22.743831 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:22.743797    2582 scope.go:117] "RemoveContainer" containerID="f993f26245235af2232b834d9bbafed60f19f17df735ff647dd32ebf544d6e44"
Jul 17 01:44:22.744228 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:22.743978    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4_odh-ai-gateway-infra(159180d3-e207-4764-9671-247894a01f3f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4" podUID="159180d3-e207-4764-9671-247894a01f3f"
Jul 17 01:44:27.467234 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.467194    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"]
Jul 17 01:44:27.467657 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.467578    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:27.467657 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.467590    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:27.467657 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.467648    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2694aa3c-90a9-4b06-a91a-0f89a02c873f" containerName="revoke-keys"
Jul 17 01:44:27.471745 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.471728    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"
Jul 17 01:44:27.477403 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.477378    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"]
Jul 17 01:44:27.564924 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.564887    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb7f12d-e310-4310-8f64-a1ec1e2a3918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl\" (UID: \"8cb7f12d-e310-4310-8f64-a1ec1e2a3918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"
Jul 17 01:44:27.666167 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.666133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb7f12d-e310-4310-8f64-a1ec1e2a3918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl\" (UID: \"8cb7f12d-e310-4310-8f64-a1ec1e2a3918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"
Jul 17 01:44:27.666792 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.666774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb7f12d-e310-4310-8f64-a1ec1e2a3918-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl\" (UID: \"8cb7f12d-e310-4310-8f64-a1ec1e2a3918\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"
Jul 17 01:44:27.784102 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.784022    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"
Jul 17 01:44:27.919217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:27.919195    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"]
Jul 17 01:44:27.921410 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:44:27.921381    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb7f12d_e310_4310_8f64_a1ec1e2a3918.slice/crio-927124166f915ddf4cb29015d8fc3d53646cd79ae3a6eb36bd4b7632d962b812 WatchSource:0}: Error finding container 927124166f915ddf4cb29015d8fc3d53646cd79ae3a6eb36bd4b7632d962b812: Status 404 returned error can't find the container with id 927124166f915ddf4cb29015d8fc3d53646cd79ae3a6eb36bd4b7632d962b812
Jul 17 01:44:28.774128 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:28.774035    2582 generic.go:356] "Generic (PLEG): container finished" podID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerID="dab8b1d0a0b29df2a3c179093bde57b79339a0de824a22939707fe9f92e7f39c" exitCode=6
Jul 17 01:44:28.774128 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:28.774070    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl" event={"ID":"8cb7f12d-e310-4310-8f64-a1ec1e2a3918","Type":"ContainerDied","Data":"dab8b1d0a0b29df2a3c179093bde57b79339a0de824a22939707fe9f92e7f39c"}
Jul 17 01:44:28.774128 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:28.774108    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl" event={"ID":"8cb7f12d-e310-4310-8f64-a1ec1e2a3918","Type":"ContainerStarted","Data":"927124166f915ddf4cb29015d8fc3d53646cd79ae3a6eb36bd4b7632d962b812"}
Jul 17 01:44:28.774708 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:28.774351    2582 scope.go:117] "RemoveContainer" containerID="dab8b1d0a0b29df2a3c179093bde57b79339a0de824a22939707fe9f92e7f39c"
Jul 17 01:44:29.780859 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:29.780826    2582 generic.go:356] "Generic (PLEG): container finished" podID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerID="a7b6ffb98751e29ce856ca69c5bb1b4c07bc9eb1e1d9b7ca68d4d531df30ebe6" exitCode=6
Jul 17 01:44:29.781274 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:29.780917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl" event={"ID":"8cb7f12d-e310-4310-8f64-a1ec1e2a3918","Type":"ContainerDied","Data":"a7b6ffb98751e29ce856ca69c5bb1b4c07bc9eb1e1d9b7ca68d4d531df30ebe6"}
Jul 17 01:44:29.781274 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:29.780974    2582 scope.go:117] "RemoveContainer" containerID="dab8b1d0a0b29df2a3c179093bde57b79339a0de824a22939707fe9f92e7f39c"
Jul 17 01:44:29.781274 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:29.781204    2582 scope.go:117] "RemoveContainer" containerID="a7b6ffb98751e29ce856ca69c5bb1b4c07bc9eb1e1d9b7ca68d4d531df30ebe6"
Jul 17 01:44:29.781470 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:29.781444    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl_odh-ai-gateway-infra(8cb7f12d-e310-4310-8f64-a1ec1e2a3918)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918"
Jul 17 01:44:30.787789 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:30.787762    2582 scope.go:117] "RemoveContainer" containerID="a7b6ffb98751e29ce856ca69c5bb1b4c07bc9eb1e1d9b7ca68d4d531df30ebe6"
Jul 17 01:44:30.788185 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:30.787941    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl_odh-ai-gateway-infra(8cb7f12d-e310-4310-8f64-a1ec1e2a3918)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918"
Jul 17 01:44:36.984952 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:36.984862    2582 scope.go:117] "RemoveContainer" containerID="f993f26245235af2232b834d9bbafed60f19f17df735ff647dd32ebf544d6e44"
Jul 17 01:44:37.821654 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:37.821622    2582 generic.go:356] "Generic (PLEG): container finished" podID="159180d3-e207-4764-9671-247894a01f3f" containerID="03b2dcafc0e50072dfe27f64e916edd00a20d8e209e79de1042204b657813f21" exitCode=6
Jul 17 01:44:37.821833 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:37.821701    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4" event={"ID":"159180d3-e207-4764-9671-247894a01f3f","Type":"ContainerDied","Data":"03b2dcafc0e50072dfe27f64e916edd00a20d8e209e79de1042204b657813f21"}
Jul 17 01:44:37.821833 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:37.821762    2582 scope.go:117] "RemoveContainer" containerID="f993f26245235af2232b834d9bbafed60f19f17df735ff647dd32ebf544d6e44"
Jul 17 01:44:37.822066 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:37.822045    2582 scope.go:117] "RemoveContainer" containerID="03b2dcafc0e50072dfe27f64e916edd00a20d8e209e79de1042204b657813f21"
Jul 17 01:44:37.822315 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:37.822294    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4_odh-ai-gateway-infra(159180d3-e207-4764-9671-247894a01f3f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4" podUID="159180d3-e207-4764-9671-247894a01f3f"
Jul 17 01:44:38.845392 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:38.845355    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"]
Jul 17 01:44:38.992207 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:38.992180    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"
Jul 17 01:44:39.068461 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:39.068408    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/159180d3-e207-4764-9671-247894a01f3f-maas-api-service-ca\") pod \"159180d3-e207-4764-9671-247894a01f3f\" (UID: \"159180d3-e207-4764-9671-247894a01f3f\") "
Jul 17 01:44:39.068803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:39.068780    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/159180d3-e207-4764-9671-247894a01f3f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "159180d3-e207-4764-9671-247894a01f3f" (UID: "159180d3-e207-4764-9671-247894a01f3f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:44:39.169733 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:39.169619    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/159180d3-e207-4764-9671-247894a01f3f-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:44:39.834779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:39.834744    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"
Jul 17 01:44:39.834779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:39.834751    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4" event={"ID":"159180d3-e207-4764-9671-247894a01f3f","Type":"ContainerDied","Data":"b24739607e62d38354f7867c648219a93a344d1c610e77513e4322731fde8600"}
Jul 17 01:44:39.835043 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:39.834841    2582 scope.go:117] "RemoveContainer" containerID="03b2dcafc0e50072dfe27f64e916edd00a20d8e209e79de1042204b657813f21"
Jul 17 01:44:39.857568 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:39.857534    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"]
Jul 17 01:44:39.862927 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:39.862900    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-v65x4"]
Jul 17 01:44:40.991480 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:40.991444    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="159180d3-e207-4764-9671-247894a01f3f" path="/var/lib/kubelet/pods/159180d3-e207-4764-9671-247894a01f3f/volumes"
Jul 17 01:44:41.984959 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:41.984929    2582 scope.go:117] "RemoveContainer" containerID="a7b6ffb98751e29ce856ca69c5bb1b4c07bc9eb1e1d9b7ca68d4d531df30ebe6"
Jul 17 01:44:42.852280 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:42.852247    2582 generic.go:356] "Generic (PLEG): container finished" podID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerID="20eda1345b5e0153d6bddba9444e31970e1320e3a0ddd517e9cf08baefcd372b" exitCode=6
Jul 17 01:44:42.852806 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:42.852328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl" event={"ID":"8cb7f12d-e310-4310-8f64-a1ec1e2a3918","Type":"ContainerDied","Data":"20eda1345b5e0153d6bddba9444e31970e1320e3a0ddd517e9cf08baefcd372b"}
Jul 17 01:44:42.852806 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:42.852393    2582 scope.go:117] "RemoveContainer" containerID="a7b6ffb98751e29ce856ca69c5bb1b4c07bc9eb1e1d9b7ca68d4d531df30ebe6"
Jul 17 01:44:42.852806 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:42.852723    2582 scope.go:117] "RemoveContainer" containerID="20eda1345b5e0153d6bddba9444e31970e1320e3a0ddd517e9cf08baefcd372b"
Jul 17 01:44:42.853004 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:42.852984    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl_odh-ai-gateway-infra(8cb7f12d-e310-4310-8f64-a1ec1e2a3918)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918"
Jul 17 01:44:43.874768 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:43.874734    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"]
Jul 17 01:44:44.011741 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.011716    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"
Jul 17 01:44:44.114853 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.114815    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb7f12d-e310-4310-8f64-a1ec1e2a3918-maas-api-service-ca\") pod \"8cb7f12d-e310-4310-8f64-a1ec1e2a3918\" (UID: \"8cb7f12d-e310-4310-8f64-a1ec1e2a3918\") "
Jul 17 01:44:44.115187 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.115165    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cb7f12d-e310-4310-8f64-a1ec1e2a3918-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8cb7f12d-e310-4310-8f64-a1ec1e2a3918" (UID: "8cb7f12d-e310-4310-8f64-a1ec1e2a3918"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:44:44.215442 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.215409    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cb7f12d-e310-4310-8f64-a1ec1e2a3918-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:44:44.864472 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.864443    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"
Jul 17 01:44:44.864678 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.864472    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl" event={"ID":"8cb7f12d-e310-4310-8f64-a1ec1e2a3918","Type":"ContainerDied","Data":"927124166f915ddf4cb29015d8fc3d53646cd79ae3a6eb36bd4b7632d962b812"}
Jul 17 01:44:44.864678 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.864555    2582 scope.go:117] "RemoveContainer" containerID="20eda1345b5e0153d6bddba9444e31970e1320e3a0ddd517e9cf08baefcd372b"
Jul 17 01:44:44.885956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.885928    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"]
Jul 17 01:44:44.890918 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.890893    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-m2njl"]
Jul 17 01:44:44.990081 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:44.990038    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" path="/var/lib/kubelet/pods/8cb7f12d-e310-4310-8f64-a1ec1e2a3918/volumes"
Jul 17 01:44:45.539475 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539442    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"]
Jul 17 01:44:45.539811 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539798    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:44:45.539852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539812    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:44:45.539852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539820    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:44:45.539852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539825    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:44:45.539852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539839    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:44:45.539852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539844    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:44:45.539852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539853    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539858    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539863    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539868    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539875    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539880    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539933    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539940    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539947    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:44:45.540032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.539952    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:44:45.543644 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.543627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"
Jul 17 01:44:45.545730 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.545709    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:44:45.545828 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.545710    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:44:45.549194 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.549168    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"]
Jul 17 01:44:45.627007 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.626969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ca50a9-19f9-41b1-95ee-f88291ff760a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g\" (UID: \"03ca50a9-19f9-41b1-95ee-f88291ff760a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"
Jul 17 01:44:45.728376 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.728333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ca50a9-19f9-41b1-95ee-f88291ff760a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g\" (UID: \"03ca50a9-19f9-41b1-95ee-f88291ff760a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"
Jul 17 01:44:45.729111 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.729090    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ca50a9-19f9-41b1-95ee-f88291ff760a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g\" (UID: \"03ca50a9-19f9-41b1-95ee-f88291ff760a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"
Jul 17 01:44:45.855661 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.855577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"
Jul 17 01:44:45.995308 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:45.995197    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"]
Jul 17 01:44:45.997933 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:44:45.997903    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ca50a9_19f9_41b1_95ee_f88291ff760a.slice/crio-1448213053ca3bada2ce3454ca02b0498f724e43231f8a0507828afb8ba9583d WatchSource:0}: Error finding container 1448213053ca3bada2ce3454ca02b0498f724e43231f8a0507828afb8ba9583d: Status 404 returned error can't find the container with id 1448213053ca3bada2ce3454ca02b0498f724e43231f8a0507828afb8ba9583d
Jul 17 01:44:46.878206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:46.878110    2582 generic.go:356] "Generic (PLEG): container finished" podID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerID="7486e27c8551ffdafbf9abe75ebfa9201a8b5bb790ce5c8dec78d5a4cb506681" exitCode=7
Jul 17 01:44:46.878206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:46.878194    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g" event={"ID":"03ca50a9-19f9-41b1-95ee-f88291ff760a","Type":"ContainerDied","Data":"7486e27c8551ffdafbf9abe75ebfa9201a8b5bb790ce5c8dec78d5a4cb506681"}
Jul 17 01:44:46.878560 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:46.878248    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g" event={"ID":"03ca50a9-19f9-41b1-95ee-f88291ff760a","Type":"ContainerStarted","Data":"1448213053ca3bada2ce3454ca02b0498f724e43231f8a0507828afb8ba9583d"}
Jul 17 01:44:46.878560 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:46.878381    2582 scope.go:117] "RemoveContainer" containerID="7486e27c8551ffdafbf9abe75ebfa9201a8b5bb790ce5c8dec78d5a4cb506681"
Jul 17 01:44:47.884426 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:47.884337    2582 generic.go:356] "Generic (PLEG): container finished" podID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerID="c6f630eb3d837f5c37c41c30e85e6f859de397b3af4dda23470c1d362d4593c8" exitCode=7
Jul 17 01:44:47.884845 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:47.884424    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g" event={"ID":"03ca50a9-19f9-41b1-95ee-f88291ff760a","Type":"ContainerDied","Data":"c6f630eb3d837f5c37c41c30e85e6f859de397b3af4dda23470c1d362d4593c8"}
Jul 17 01:44:47.884845 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:47.884540    2582 scope.go:117] "RemoveContainer" containerID="7486e27c8551ffdafbf9abe75ebfa9201a8b5bb790ce5c8dec78d5a4cb506681"
Jul 17 01:44:47.884845 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:47.884782    2582 scope.go:117] "RemoveContainer" containerID="c6f630eb3d837f5c37c41c30e85e6f859de397b3af4dda23470c1d362d4593c8"
Jul 17 01:44:47.885031 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:47.885012    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g_odh-ai-gateway-infra(03ca50a9-19f9-41b1-95ee-f88291ff760a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a"
Jul 17 01:44:48.891625 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:48.891597    2582 scope.go:117] "RemoveContainer" containerID="c6f630eb3d837f5c37c41c30e85e6f859de397b3af4dda23470c1d362d4593c8"
Jul 17 01:44:48.892012 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:48.891794    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g_odh-ai-gateway-infra(03ca50a9-19f9-41b1-95ee-f88291ff760a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a"
Jul 17 01:44:58.985406 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:58.985373    2582 scope.go:117] "RemoveContainer" containerID="c6f630eb3d837f5c37c41c30e85e6f859de397b3af4dda23470c1d362d4593c8"
Jul 17 01:44:59.947526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:59.947473    2582 generic.go:356] "Generic (PLEG): container finished" podID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerID="6740c05c838e4ec16a86464c6b7937aae48d502ed08628e93e8870a6f2e1ff56" exitCode=7
Jul 17 01:44:59.947526 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:59.947522    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g" event={"ID":"03ca50a9-19f9-41b1-95ee-f88291ff760a","Type":"ContainerDied","Data":"6740c05c838e4ec16a86464c6b7937aae48d502ed08628e93e8870a6f2e1ff56"}
Jul 17 01:44:59.947789 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:59.947583    2582 scope.go:117] "RemoveContainer" containerID="c6f630eb3d837f5c37c41c30e85e6f859de397b3af4dda23470c1d362d4593c8"
Jul 17 01:44:59.947933 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:44:59.947917    2582 scope.go:117] "RemoveContainer" containerID="6740c05c838e4ec16a86464c6b7937aae48d502ed08628e93e8870a6f2e1ff56"
Jul 17 01:44:59.948180 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:44:59.948159    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g_odh-ai-gateway-infra(03ca50a9-19f9-41b1-95ee-f88291ff760a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a"
Jul 17 01:45:00.134598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.134521    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"]
Jul 17 01:45:00.134974 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.134949    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="159180d3-e207-4764-9671-247894a01f3f" containerName="revoke-keys"
Jul 17 01:45:00.134974 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.134963    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb7f12d-e310-4310-8f64-a1ec1e2a3918" containerName="revoke-keys"
Jul 17 01:45:00.138938 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.138912    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"
Jul 17 01:45:00.144657 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.144633    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"]
Jul 17 01:45:00.151660 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.151628    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"
Jul 17 01:45:00.285349 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.285326    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"]
Jul 17 01:45:00.287448 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:45:00.287418    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77077121_8255_4a50_b674_49b38d59ea8e.slice/crio-c370ed5aca59e1e7db80ed8dbebcf1c4d8cf60a4973d955980b3160d49b17c10 WatchSource:0}: Error finding container c370ed5aca59e1e7db80ed8dbebcf1c4d8cf60a4973d955980b3160d49b17c10: Status 404 returned error can't find the container with id c370ed5aca59e1e7db80ed8dbebcf1c4d8cf60a4973d955980b3160d49b17c10
Jul 17 01:45:00.955105 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.955074    2582 generic.go:356] "Generic (PLEG): container finished" podID="77077121-8255-4a50-b674-49b38d59ea8e" containerID="80fe57b77c29e6b4609a176d4523593cfbcf607d9fc4a9c7ba78b50514f4bdc1" exitCode=7
Jul 17 01:45:00.955280 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.955198    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9" event={"ID":"77077121-8255-4a50-b674-49b38d59ea8e","Type":"ContainerDied","Data":"80fe57b77c29e6b4609a176d4523593cfbcf607d9fc4a9c7ba78b50514f4bdc1"}
Jul 17 01:45:00.955280 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.955252    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9" event={"ID":"77077121-8255-4a50-b674-49b38d59ea8e","Type":"ContainerStarted","Data":"c370ed5aca59e1e7db80ed8dbebcf1c4d8cf60a4973d955980b3160d49b17c10"}
Jul 17 01:45:00.955280 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.955264    2582 scope.go:117] "RemoveContainer" containerID="80fe57b77c29e6b4609a176d4523593cfbcf607d9fc4a9c7ba78b50514f4bdc1"
Jul 17 01:45:00.974549 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:00.974524    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"]
Jul 17 01:45:01.099911 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.099889    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"
Jul 17 01:45:01.168994 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.168962    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ca50a9-19f9-41b1-95ee-f88291ff760a-maas-api-service-ca\") pod \"03ca50a9-19f9-41b1-95ee-f88291ff760a\" (UID: \"03ca50a9-19f9-41b1-95ee-f88291ff760a\") "
Jul 17 01:45:01.169407 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.169375    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03ca50a9-19f9-41b1-95ee-f88291ff760a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "03ca50a9-19f9-41b1-95ee-f88291ff760a" (UID: "03ca50a9-19f9-41b1-95ee-f88291ff760a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:45:01.270453 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.270420    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ca50a9-19f9-41b1-95ee-f88291ff760a-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:45:01.961547 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.961518    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"
Jul 17 01:45:01.961547 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.961520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g" event={"ID":"03ca50a9-19f9-41b1-95ee-f88291ff760a","Type":"ContainerDied","Data":"1448213053ca3bada2ce3454ca02b0498f724e43231f8a0507828afb8ba9583d"}
Jul 17 01:45:01.961814 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.961599    2582 scope.go:117] "RemoveContainer" containerID="6740c05c838e4ec16a86464c6b7937aae48d502ed08628e93e8870a6f2e1ff56"
Jul 17 01:45:01.963509 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.963465    2582 generic.go:356] "Generic (PLEG): container finished" podID="77077121-8255-4a50-b674-49b38d59ea8e" containerID="318037bebe96888e1d3597c7bdbe9b14d568f58b51169b6a7d244ec6188c2033" exitCode=7
Jul 17 01:45:01.963626 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.963529    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9" event={"ID":"77077121-8255-4a50-b674-49b38d59ea8e","Type":"ContainerDied","Data":"318037bebe96888e1d3597c7bdbe9b14d568f58b51169b6a7d244ec6188c2033"}
Jul 17 01:45:01.963724 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.963711    2582 scope.go:117] "RemoveContainer" containerID="318037bebe96888e1d3597c7bdbe9b14d568f58b51169b6a7d244ec6188c2033"
Jul 17 01:45:01.963923 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:45:01.963906    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9_odh-ai-gateway-infra(77077121-8255-4a50-b674-49b38d59ea8e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9" podUID="77077121-8255-4a50-b674-49b38d59ea8e"
Jul 17 01:45:01.971571 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.971477    2582 scope.go:117] "RemoveContainer" containerID="80fe57b77c29e6b4609a176d4523593cfbcf607d9fc4a9c7ba78b50514f4bdc1"
Jul 17 01:45:01.994965 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:01.994939    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"]
Jul 17 01:45:02.000921 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:02.000897    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-8fv7g"]
Jul 17 01:45:02.971143 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:02.971117    2582 scope.go:117] "RemoveContainer" containerID="318037bebe96888e1d3597c7bdbe9b14d568f58b51169b6a7d244ec6188c2033"
Jul 17 01:45:02.971577 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:45:02.971332    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9_odh-ai-gateway-infra(77077121-8255-4a50-b674-49b38d59ea8e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9" podUID="77077121-8255-4a50-b674-49b38d59ea8e"
Jul 17 01:45:02.990127 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:02.990096    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" path="/var/lib/kubelet/pods/03ca50a9-19f9-41b1-95ee-f88291ff760a/volumes"
Jul 17 01:45:15.984674 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:15.984642    2582 scope.go:117] "RemoveContainer" containerID="318037bebe96888e1d3597c7bdbe9b14d568f58b51169b6a7d244ec6188c2033"
Jul 17 01:45:17.037319 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:17.037223    2582 generic.go:356] "Generic (PLEG): container finished" podID="77077121-8255-4a50-b674-49b38d59ea8e" containerID="de18fd8cbd7ce2e2b4d2870c1db6f7b94b8aad3ca8a20e8af162da81410a2b18" exitCode=7
Jul 17 01:45:17.037319 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:17.037295    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9" event={"ID":"77077121-8255-4a50-b674-49b38d59ea8e","Type":"ContainerDied","Data":"de18fd8cbd7ce2e2b4d2870c1db6f7b94b8aad3ca8a20e8af162da81410a2b18"}
Jul 17 01:45:17.037796 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:17.037361    2582 scope.go:117] "RemoveContainer" containerID="318037bebe96888e1d3597c7bdbe9b14d568f58b51169b6a7d244ec6188c2033"
Jul 17 01:45:17.037796 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:17.037656    2582 scope.go:117] "RemoveContainer" containerID="de18fd8cbd7ce2e2b4d2870c1db6f7b94b8aad3ca8a20e8af162da81410a2b18"
Jul 17 01:45:17.037898 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:45:17.037877    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9_odh-ai-gateway-infra(77077121-8255-4a50-b674-49b38d59ea8e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9" podUID="77077121-8255-4a50-b674-49b38d59ea8e"
Jul 17 01:45:18.060581 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:18.060545    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"]
Jul 17 01:45:18.200091 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:18.200060    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"
Jul 17 01:45:19.048573 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.048539    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"
Jul 17 01:45:19.048573 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.048562    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9" event={"ID":"77077121-8255-4a50-b674-49b38d59ea8e","Type":"ContainerDied","Data":"c370ed5aca59e1e7db80ed8dbebcf1c4d8cf60a4973d955980b3160d49b17c10"}
Jul 17 01:45:19.048847 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.048635    2582 scope.go:117] "RemoveContainer" containerID="de18fd8cbd7ce2e2b4d2870c1db6f7b94b8aad3ca8a20e8af162da81410a2b18"
Jul 17 01:45:19.063275 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.063251    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"]
Jul 17 01:45:19.069657 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.069630    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-2ef2c65d-29737545-zg9v9"]
Jul 17 01:45:19.490849 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.490814    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"]
Jul 17 01:45:19.491237 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491220    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:45:19.491333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491241    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:45:19.491333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491257    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.491333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491266    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.491333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491283    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.491333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491291    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.491333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491308    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:45:19.491333 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491316    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:45:19.491723 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491335    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.491723 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491344    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.491723 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491440    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:45:19.491723 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491451    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:45:19.491723 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491463    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.491723 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491473    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.491723 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.491483    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="77077121-8255-4a50-b674-49b38d59ea8e" containerName="cleanup"
Jul 17 01:45:19.496032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.496012    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"
Jul 17 01:45:19.498963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.498938    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:45:19.499085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.499007    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:45:19.500405 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.500384    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"]
Jul 17 01:45:19.528961 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.528929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b7144a1-45a7-451e-a7f8-7331f16ef32a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq\" (UID: \"2b7144a1-45a7-451e-a7f8-7331f16ef32a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"
Jul 17 01:45:19.630174 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.630132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b7144a1-45a7-451e-a7f8-7331f16ef32a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq\" (UID: \"2b7144a1-45a7-451e-a7f8-7331f16ef32a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"
Jul 17 01:45:19.630821 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.630795    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b7144a1-45a7-451e-a7f8-7331f16ef32a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq\" (UID: \"2b7144a1-45a7-451e-a7f8-7331f16ef32a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"
Jul 17 01:45:19.807739 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.807639    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"
Jul 17 01:45:19.942391 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:19.942366    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"]
Jul 17 01:45:19.944715 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:45:19.944685    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b7144a1_45a7_451e_a7f8_7331f16ef32a.slice/crio-86aa75574b913b56cccb6707ec7f066d91dc76fe3edaed8fa5ba1ed4a938657e WatchSource:0}: Error finding container 86aa75574b913b56cccb6707ec7f066d91dc76fe3edaed8fa5ba1ed4a938657e: Status 404 returned error can't find the container with id 86aa75574b913b56cccb6707ec7f066d91dc76fe3edaed8fa5ba1ed4a938657e
Jul 17 01:45:20.054105 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:20.054066    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq" event={"ID":"2b7144a1-45a7-451e-a7f8-7331f16ef32a","Type":"ContainerStarted","Data":"86aa75574b913b56cccb6707ec7f066d91dc76fe3edaed8fa5ba1ed4a938657e"}
Jul 17 01:45:20.991297 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:20.991261    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77077121-8255-4a50-b674-49b38d59ea8e" path="/var/lib/kubelet/pods/77077121-8255-4a50-b674-49b38d59ea8e/volumes"
Jul 17 01:45:21.061447 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:21.061416    2582 generic.go:356] "Generic (PLEG): container finished" podID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerID="cb1f60525d89d387de118b11b6a47f85311518e5d4290dfcdb0ece1aecc2739e" exitCode=6
Jul 17 01:45:21.061644 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:21.061525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq" event={"ID":"2b7144a1-45a7-451e-a7f8-7331f16ef32a","Type":"ContainerDied","Data":"cb1f60525d89d387de118b11b6a47f85311518e5d4290dfcdb0ece1aecc2739e"}
Jul 17 01:45:21.061704 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:21.061690    2582 scope.go:117] "RemoveContainer" containerID="cb1f60525d89d387de118b11b6a47f85311518e5d4290dfcdb0ece1aecc2739e"
Jul 17 01:45:22.068709 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:22.068675    2582 generic.go:356] "Generic (PLEG): container finished" podID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerID="3396bbfd01ab9a305c5444744828a18b28a459c944d582f2eb8718e2c26eccc0" exitCode=6
Jul 17 01:45:22.069146 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:22.068740    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq" event={"ID":"2b7144a1-45a7-451e-a7f8-7331f16ef32a","Type":"ContainerDied","Data":"3396bbfd01ab9a305c5444744828a18b28a459c944d582f2eb8718e2c26eccc0"}
Jul 17 01:45:22.069146 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:22.068820    2582 scope.go:117] "RemoveContainer" containerID="cb1f60525d89d387de118b11b6a47f85311518e5d4290dfcdb0ece1aecc2739e"
Jul 17 01:45:22.069146 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:22.069029    2582 scope.go:117] "RemoveContainer" containerID="3396bbfd01ab9a305c5444744828a18b28a459c944d582f2eb8718e2c26eccc0"
Jul 17 01:45:22.069261 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:45:22.069234    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq_odh-ai-gateway-infra(2b7144a1-45a7-451e-a7f8-7331f16ef32a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a"
Jul 17 01:45:23.074864 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:23.074836    2582 scope.go:117] "RemoveContainer" containerID="3396bbfd01ab9a305c5444744828a18b28a459c944d582f2eb8718e2c26eccc0"
Jul 17 01:45:23.075269 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:45:23.075025    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq_odh-ai-gateway-infra(2b7144a1-45a7-451e-a7f8-7331f16ef32a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a"
Jul 17 01:45:34.984691 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:34.984651    2582 scope.go:117] "RemoveContainer" containerID="3396bbfd01ab9a305c5444744828a18b28a459c944d582f2eb8718e2c26eccc0"
Jul 17 01:45:36.136279 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:36.136244    2582 generic.go:356] "Generic (PLEG): container finished" podID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerID="acda9e1b9da88cb7e91080212b7480fdbfc4c22192da20f6e031a159093b31ed" exitCode=6
Jul 17 01:45:36.136730 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:36.136315    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq" event={"ID":"2b7144a1-45a7-451e-a7f8-7331f16ef32a","Type":"ContainerDied","Data":"acda9e1b9da88cb7e91080212b7480fdbfc4c22192da20f6e031a159093b31ed"}
Jul 17 01:45:36.136730 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:36.136377    2582 scope.go:117] "RemoveContainer" containerID="3396bbfd01ab9a305c5444744828a18b28a459c944d582f2eb8718e2c26eccc0"
Jul 17 01:45:36.136730 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:36.136655    2582 scope.go:117] "RemoveContainer" containerID="acda9e1b9da88cb7e91080212b7480fdbfc4c22192da20f6e031a159093b31ed"
Jul 17 01:45:36.136906 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:45:36.136887    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq_odh-ai-gateway-infra(2b7144a1-45a7-451e-a7f8-7331f16ef32a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a"
Jul 17 01:45:37.162359 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:37.162325    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"]
Jul 17 01:45:37.302940 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:37.302911    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"
Jul 17 01:45:37.384082 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:37.384046    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b7144a1-45a7-451e-a7f8-7331f16ef32a-maas-api-service-ca\") pod \"2b7144a1-45a7-451e-a7f8-7331f16ef32a\" (UID: \"2b7144a1-45a7-451e-a7f8-7331f16ef32a\") "
Jul 17 01:45:37.384411 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:37.384388    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b7144a1-45a7-451e-a7f8-7331f16ef32a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2b7144a1-45a7-451e-a7f8-7331f16ef32a" (UID: "2b7144a1-45a7-451e-a7f8-7331f16ef32a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:45:37.484654 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:37.484621    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b7144a1-45a7-451e-a7f8-7331f16ef32a-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:45:38.148085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:38.148052    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq" event={"ID":"2b7144a1-45a7-451e-a7f8-7331f16ef32a","Type":"ContainerDied","Data":"86aa75574b913b56cccb6707ec7f066d91dc76fe3edaed8fa5ba1ed4a938657e"}
Jul 17 01:45:38.148085 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:38.148087    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"
Jul 17 01:45:38.148313 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:38.148123    2582 scope.go:117] "RemoveContainer" containerID="acda9e1b9da88cb7e91080212b7480fdbfc4c22192da20f6e031a159093b31ed"
Jul 17 01:45:38.178371 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:38.178338    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"]
Jul 17 01:45:38.181302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:38.181279    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-4gtnq"]
Jul 17 01:45:38.991605 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:45:38.991570    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" path="/var/lib/kubelet/pods/2b7144a1-45a7-451e-a7f8-7331f16ef32a/volumes"
Jul 17 01:46:09.529319 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529237    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"]
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529606    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529619    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529629    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529636    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529644    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529650    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529658    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529663    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529723    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529730    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:09.529779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.529737    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="03ca50a9-19f9-41b1-95ee-f88291ff760a" containerName="revoke-keys"
Jul 17 01:46:09.532838 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.532822    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"
Jul 17 01:46:09.535020 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.535001    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:46:09.535126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.535017    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:46:09.538819 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.538793    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"]
Jul 17 01:46:09.556715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.556683    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6de7ba-95ff-4229-a327-a11be3de920a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc\" (UID: \"fc6de7ba-95ff-4229-a327-a11be3de920a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"
Jul 17 01:46:09.657267 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.657206    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6de7ba-95ff-4229-a327-a11be3de920a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc\" (UID: \"fc6de7ba-95ff-4229-a327-a11be3de920a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"
Jul 17 01:46:09.657895 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.657874    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6de7ba-95ff-4229-a327-a11be3de920a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc\" (UID: \"fc6de7ba-95ff-4229-a327-a11be3de920a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"
Jul 17 01:46:09.844914 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.844833    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"
Jul 17 01:46:09.978652 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:09.978626    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"]
Jul 17 01:46:09.980595 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:46:09.980567    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6de7ba_95ff_4229_a327_a11be3de920a.slice/crio-f2a4cdac135fa7d8be83924a9a15cc2c227656534e5065f757572d0ba0f01ee4 WatchSource:0}: Error finding container f2a4cdac135fa7d8be83924a9a15cc2c227656534e5065f757572d0ba0f01ee4: Status 404 returned error can't find the container with id f2a4cdac135fa7d8be83924a9a15cc2c227656534e5065f757572d0ba0f01ee4
Jul 17 01:46:10.301383 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:10.301347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" event={"ID":"fc6de7ba-95ff-4229-a327-a11be3de920a","Type":"ContainerStarted","Data":"f2a4cdac135fa7d8be83924a9a15cc2c227656534e5065f757572d0ba0f01ee4"}
Jul 17 01:46:11.308578 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:11.308545    2582 generic.go:356] "Generic (PLEG): container finished" podID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerID="7b2320a9ab0e9f171aef514dcabf520ab0543da47b4a7c23cdafe182c26a2597" exitCode=6
Jul 17 01:46:11.309068 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:11.308631    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" event={"ID":"fc6de7ba-95ff-4229-a327-a11be3de920a","Type":"ContainerDied","Data":"7b2320a9ab0e9f171aef514dcabf520ab0543da47b4a7c23cdafe182c26a2597"}
Jul 17 01:46:11.309068 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:11.308809    2582 scope.go:117] "RemoveContainer" containerID="7b2320a9ab0e9f171aef514dcabf520ab0543da47b4a7c23cdafe182c26a2597"
Jul 17 01:46:12.318772 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:12.318686    2582 generic.go:356] "Generic (PLEG): container finished" podID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerID="05f7a060b6c972af07c1531e0b4211c03e4f10a27fff191eda60e3f263ab4df6" exitCode=6
Jul 17 01:46:12.319169 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:12.318774    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" event={"ID":"fc6de7ba-95ff-4229-a327-a11be3de920a","Type":"ContainerDied","Data":"05f7a060b6c972af07c1531e0b4211c03e4f10a27fff191eda60e3f263ab4df6"}
Jul 17 01:46:12.319169 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:12.318840    2582 scope.go:117] "RemoveContainer" containerID="7b2320a9ab0e9f171aef514dcabf520ab0543da47b4a7c23cdafe182c26a2597"
Jul 17 01:46:12.319169 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:12.319083    2582 scope.go:117] "RemoveContainer" containerID="05f7a060b6c972af07c1531e0b4211c03e4f10a27fff191eda60e3f263ab4df6"
Jul 17 01:46:12.319307 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:46:12.319289    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc_odh-ai-gateway-infra(fc6de7ba-95ff-4229-a327-a11be3de920a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a"
Jul 17 01:46:13.325083 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:13.325052    2582 scope.go:117] "RemoveContainer" containerID="05f7a060b6c972af07c1531e0b4211c03e4f10a27fff191eda60e3f263ab4df6"
Jul 17 01:46:13.325465 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:46:13.325279    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc_odh-ai-gateway-infra(fc6de7ba-95ff-4229-a327-a11be3de920a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a"
Jul 17 01:46:17.553027 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:17.552992    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"]
Jul 17 01:46:17.553474 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:17.553422    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b7144a1-45a7-451e-a7f8-7331f16ef32a" containerName="revoke-keys"
Jul 17 01:46:17.556521 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:17.556483    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"
Jul 17 01:46:17.563239 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:17.563215    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"]
Jul 17 01:46:17.630285 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:17.630251    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595c0a1b-5895-4cb3-ba9c-fab0103bb682-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4\" (UID: \"595c0a1b-5895-4cb3-ba9c-fab0103bb682\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"
Jul 17 01:46:17.731315 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:17.731280    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595c0a1b-5895-4cb3-ba9c-fab0103bb682-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4\" (UID: \"595c0a1b-5895-4cb3-ba9c-fab0103bb682\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"
Jul 17 01:46:17.732005 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:17.731982    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595c0a1b-5895-4cb3-ba9c-fab0103bb682-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4\" (UID: \"595c0a1b-5895-4cb3-ba9c-fab0103bb682\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"
Jul 17 01:46:17.867763 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:17.867676    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"
Jul 17 01:46:18.003276 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:18.003130    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"]
Jul 17 01:46:18.005619 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:46:18.005589    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595c0a1b_5895_4cb3_ba9c_fab0103bb682.slice/crio-ff853a3f1da26bbf487a76c5d0ec92ee9b0bd8059554c9549898df3411995b77 WatchSource:0}: Error finding container ff853a3f1da26bbf487a76c5d0ec92ee9b0bd8059554c9549898df3411995b77: Status 404 returned error can't find the container with id ff853a3f1da26bbf487a76c5d0ec92ee9b0bd8059554c9549898df3411995b77
Jul 17 01:46:18.350182 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:18.350147    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" event={"ID":"595c0a1b-5895-4cb3-ba9c-fab0103bb682","Type":"ContainerStarted","Data":"43f21205bbd620afb22cb0bdc6e626acc8b3f456e6aba933269e123d1dffc852"}
Jul 17 01:46:18.350354 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:18.350198    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" event={"ID":"595c0a1b-5895-4cb3-ba9c-fab0103bb682","Type":"ContainerStarted","Data":"ff853a3f1da26bbf487a76c5d0ec92ee9b0bd8059554c9549898df3411995b77"}
Jul 17 01:46:18.364952 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:18.364906    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" podStartSLOduration=1.36489079 podStartE2EDuration="1.36489079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:46:18.363394912 +0000 UTC m=+1985.867080773" watchObservedRunningTime="2026-07-17 01:46:18.36489079 +0000 UTC m=+1985.868576651"
Jul 17 01:46:19.356162 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:19.356124    2582 generic.go:356] "Generic (PLEG): container finished" podID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerID="43f21205bbd620afb22cb0bdc6e626acc8b3f456e6aba933269e123d1dffc852" exitCode=6
Jul 17 01:46:19.356691 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:19.356179    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" event={"ID":"595c0a1b-5895-4cb3-ba9c-fab0103bb682","Type":"ContainerDied","Data":"43f21205bbd620afb22cb0bdc6e626acc8b3f456e6aba933269e123d1dffc852"}
Jul 17 01:46:19.356691 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:19.356445    2582 scope.go:117] "RemoveContainer" containerID="43f21205bbd620afb22cb0bdc6e626acc8b3f456e6aba933269e123d1dffc852"
Jul 17 01:46:20.362293 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:20.362259    2582 generic.go:356] "Generic (PLEG): container finished" podID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerID="5207d284d74177f8da8249b079657f8f4e5adfdcbfc5b2bbda6a83fd447a6262" exitCode=6
Jul 17 01:46:20.362733 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:20.362344    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" event={"ID":"595c0a1b-5895-4cb3-ba9c-fab0103bb682","Type":"ContainerDied","Data":"5207d284d74177f8da8249b079657f8f4e5adfdcbfc5b2bbda6a83fd447a6262"}
Jul 17 01:46:20.362733 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:20.362401    2582 scope.go:117] "RemoveContainer" containerID="43f21205bbd620afb22cb0bdc6e626acc8b3f456e6aba933269e123d1dffc852"
Jul 17 01:46:20.362733 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:20.362521    2582 scope.go:117] "RemoveContainer" containerID="5207d284d74177f8da8249b079657f8f4e5adfdcbfc5b2bbda6a83fd447a6262"
Jul 17 01:46:20.362733 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:46:20.362718    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4_odh-ai-gateway-infra(595c0a1b-5895-4cb3-ba9c-fab0103bb682)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682"
Jul 17 01:46:21.369170 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:21.369145    2582 scope.go:117] "RemoveContainer" containerID="5207d284d74177f8da8249b079657f8f4e5adfdcbfc5b2bbda6a83fd447a6262"
Jul 17 01:46:21.369546 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:46:21.369335    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4_odh-ai-gateway-infra(595c0a1b-5895-4cb3-ba9c-fab0103bb682)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682"
Jul 17 01:46:23.985209 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:23.985181    2582 scope.go:117] "RemoveContainer" containerID="05f7a060b6c972af07c1531e0b4211c03e4f10a27fff191eda60e3f263ab4df6"
Jul 17 01:46:24.383776 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:24.383743    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" event={"ID":"fc6de7ba-95ff-4229-a327-a11be3de920a","Type":"ContainerStarted","Data":"7580994c0d4c2d18729f5d8d732334354258560bcdc6f676771afad562756e71"}
Jul 17 01:46:24.399208 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:24.399162    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" podStartSLOduration=15.399147235 podStartE2EDuration="15.399147235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:46:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:46:24.397477505 +0000 UTC m=+1991.901163367" watchObservedRunningTime="2026-07-17 01:46:24.399147235 +0000 UTC m=+1991.902833096"
Jul 17 01:46:25.390450 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:25.390418    2582 generic.go:356] "Generic (PLEG): container finished" podID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerID="7580994c0d4c2d18729f5d8d732334354258560bcdc6f676771afad562756e71" exitCode=6
Jul 17 01:46:25.390892 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:25.390513    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" event={"ID":"fc6de7ba-95ff-4229-a327-a11be3de920a","Type":"ContainerDied","Data":"7580994c0d4c2d18729f5d8d732334354258560bcdc6f676771afad562756e71"}
Jul 17 01:46:25.390892 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:25.390581    2582 scope.go:117] "RemoveContainer" containerID="05f7a060b6c972af07c1531e0b4211c03e4f10a27fff191eda60e3f263ab4df6"
Jul 17 01:46:25.390892 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:25.390818    2582 scope.go:117] "RemoveContainer" containerID="7580994c0d4c2d18729f5d8d732334354258560bcdc6f676771afad562756e71"
Jul 17 01:46:25.391078 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:46:25.391054    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc_odh-ai-gateway-infra(fc6de7ba-95ff-4229-a327-a11be3de920a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a"
Jul 17 01:46:25.405948 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:25.405926    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"]
Jul 17 01:46:26.533242 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:26.533218    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"
Jul 17 01:46:26.612675 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:26.612646    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6de7ba-95ff-4229-a327-a11be3de920a-maas-api-service-ca\") pod \"fc6de7ba-95ff-4229-a327-a11be3de920a\" (UID: \"fc6de7ba-95ff-4229-a327-a11be3de920a\") "
Jul 17 01:46:26.613014 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:26.612993    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc6de7ba-95ff-4229-a327-a11be3de920a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc6de7ba-95ff-4229-a327-a11be3de920a" (UID: "fc6de7ba-95ff-4229-a327-a11be3de920a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:46:26.713935 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:26.713889    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc6de7ba-95ff-4229-a327-a11be3de920a-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:46:27.402802 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:27.402709    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc" event={"ID":"fc6de7ba-95ff-4229-a327-a11be3de920a","Type":"ContainerDied","Data":"f2a4cdac135fa7d8be83924a9a15cc2c227656534e5065f757572d0ba0f01ee4"}
Jul 17 01:46:27.402802 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:27.402782    2582 scope.go:117] "RemoveContainer" containerID="7580994c0d4c2d18729f5d8d732334354258560bcdc6f676771afad562756e71"
Jul 17 01:46:27.403022 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:27.402729    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"
Jul 17 01:46:27.417958 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:27.417934    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"]
Jul 17 01:46:27.423174 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:27.423151    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-jbltc"]
Jul 17 01:46:28.989836 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:28.989805    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" path="/var/lib/kubelet/pods/fc6de7ba-95ff-4229-a327-a11be3de920a/volumes"
Jul 17 01:46:32.986760 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:32.986734    2582 scope.go:117] "RemoveContainer" containerID="5207d284d74177f8da8249b079657f8f4e5adfdcbfc5b2bbda6a83fd447a6262"
Jul 17 01:46:33.433723 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:33.433687    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" event={"ID":"595c0a1b-5895-4cb3-ba9c-fab0103bb682","Type":"ContainerStarted","Data":"64e994d24dc216c942bf3ea39de563d0d84d8ae524e114c6ff83720c7260bdff"}
Jul 17 01:46:34.444103 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:34.441787    2582 generic.go:356] "Generic (PLEG): container finished" podID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerID="64e994d24dc216c942bf3ea39de563d0d84d8ae524e114c6ff83720c7260bdff" exitCode=6
Jul 17 01:46:34.444103 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:34.441856    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" event={"ID":"595c0a1b-5895-4cb3-ba9c-fab0103bb682","Type":"ContainerDied","Data":"64e994d24dc216c942bf3ea39de563d0d84d8ae524e114c6ff83720c7260bdff"}
Jul 17 01:46:34.444103 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:34.441951    2582 scope.go:117] "RemoveContainer" containerID="5207d284d74177f8da8249b079657f8f4e5adfdcbfc5b2bbda6a83fd447a6262"
Jul 17 01:46:34.444103 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:34.442355    2582 scope.go:117] "RemoveContainer" containerID="64e994d24dc216c942bf3ea39de563d0d84d8ae524e114c6ff83720c7260bdff"
Jul 17 01:46:34.444103 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:46:34.442726    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4_odh-ai-gateway-infra(595c0a1b-5895-4cb3-ba9c-fab0103bb682)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682"
Jul 17 01:46:34.472096 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:34.472064    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"]
Jul 17 01:46:35.585671 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:35.585643    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"
Jul 17 01:46:35.696027 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:35.695986    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595c0a1b-5895-4cb3-ba9c-fab0103bb682-maas-api-service-ca\") pod \"595c0a1b-5895-4cb3-ba9c-fab0103bb682\" (UID: \"595c0a1b-5895-4cb3-ba9c-fab0103bb682\") "
Jul 17 01:46:35.696389 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:35.696366    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/595c0a1b-5895-4cb3-ba9c-fab0103bb682-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "595c0a1b-5895-4cb3-ba9c-fab0103bb682" (UID: "595c0a1b-5895-4cb3-ba9c-fab0103bb682"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:46:35.797557 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:35.797452    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/595c0a1b-5895-4cb3-ba9c-fab0103bb682-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:46:36.455004 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:36.454971    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"
Jul 17 01:46:36.455004 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:36.454993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4" event={"ID":"595c0a1b-5895-4cb3-ba9c-fab0103bb682","Type":"ContainerDied","Data":"ff853a3f1da26bbf487a76c5d0ec92ee9b0bd8059554c9549898df3411995b77"}
Jul 17 01:46:36.455239 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:36.455058    2582 scope.go:117] "RemoveContainer" containerID="64e994d24dc216c942bf3ea39de563d0d84d8ae524e114c6ff83720c7260bdff"
Jul 17 01:46:36.474684 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:36.474656    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"]
Jul 17 01:46:36.480701 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:36.480665    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-fdkr4"]
Jul 17 01:46:36.991039 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:36.991005    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" path="/var/lib/kubelet/pods/595c0a1b-5895-4cb3-ba9c-fab0103bb682/volumes"
Jul 17 01:46:59.566675 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.566632    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"]
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.566977    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.566987    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.566998    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567004    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567011    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567016    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567038    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567042    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567113    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567120    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567128    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567135    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:46:59.567181 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.567140    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:46:59.570133 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.570118    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"
Jul 17 01:46:59.572350 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.572330    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:46:59.572459 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.572418    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:46:59.578721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.578696    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"]
Jul 17 01:46:59.606842 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.606809    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7de43a-7910-485f-bc1f-29abeb32afae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt\" (UID: \"6d7de43a-7910-485f-bc1f-29abeb32afae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"
Jul 17 01:46:59.708139 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.708101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7de43a-7910-485f-bc1f-29abeb32afae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt\" (UID: \"6d7de43a-7910-485f-bc1f-29abeb32afae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"
Jul 17 01:46:59.708767 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.708746    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7de43a-7910-485f-bc1f-29abeb32afae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt\" (UID: \"6d7de43a-7910-485f-bc1f-29abeb32afae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"
Jul 17 01:46:59.882280 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:46:59.882201    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"
Jul 17 01:47:00.017648 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:00.017618    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"]
Jul 17 01:47:00.021321 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:47:00.021292    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7de43a_7910_485f_bc1f_29abeb32afae.slice/crio-c783488e5b42c3a203decb044faae2888185eb24d06b6bad201ecac48fbe2f61 WatchSource:0}: Error finding container c783488e5b42c3a203decb044faae2888185eb24d06b6bad201ecac48fbe2f61: Status 404 returned error can't find the container with id c783488e5b42c3a203decb044faae2888185eb24d06b6bad201ecac48fbe2f61
Jul 17 01:47:00.564958 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:00.564871    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" event={"ID":"6d7de43a-7910-485f-bc1f-29abeb32afae","Type":"ContainerStarted","Data":"c194d57a00e61fddfd61e6f12d6ade6d2ac913e0f9e43e22592159b982c2d1df"}
Jul 17 01:47:00.564958 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:00.564927    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" event={"ID":"6d7de43a-7910-485f-bc1f-29abeb32afae","Type":"ContainerStarted","Data":"c783488e5b42c3a203decb044faae2888185eb24d06b6bad201ecac48fbe2f61"}
Jul 17 01:47:00.582034 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:00.581990    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" podStartSLOduration=1.5819732100000001 podStartE2EDuration="1.58197321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:46:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:47:00.580428367 +0000 UTC m=+2028.084114229" watchObservedRunningTime="2026-07-17 01:47:00.58197321 +0000 UTC m=+2028.085659072"
Jul 17 01:47:01.571371 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:01.571331    2582 generic.go:356] "Generic (PLEG): container finished" podID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerID="c194d57a00e61fddfd61e6f12d6ade6d2ac913e0f9e43e22592159b982c2d1df" exitCode=6
Jul 17 01:47:01.571586 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:01.571422    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" event={"ID":"6d7de43a-7910-485f-bc1f-29abeb32afae","Type":"ContainerDied","Data":"c194d57a00e61fddfd61e6f12d6ade6d2ac913e0f9e43e22592159b982c2d1df"}
Jul 17 01:47:01.571636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:01.571611    2582 scope.go:117] "RemoveContainer" containerID="c194d57a00e61fddfd61e6f12d6ade6d2ac913e0f9e43e22592159b982c2d1df"
Jul 17 01:47:02.578303 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:02.578210    2582 generic.go:356] "Generic (PLEG): container finished" podID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerID="1b3ef908af70483685199fe2c8293b699b088a9fd748b78e814a2ede48dfafb3" exitCode=6
Jul 17 01:47:02.578303 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:02.578269    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" event={"ID":"6d7de43a-7910-485f-bc1f-29abeb32afae","Type":"ContainerDied","Data":"1b3ef908af70483685199fe2c8293b699b088a9fd748b78e814a2ede48dfafb3"}
Jul 17 01:47:02.578779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:02.578330    2582 scope.go:117] "RemoveContainer" containerID="c194d57a00e61fddfd61e6f12d6ade6d2ac913e0f9e43e22592159b982c2d1df"
Jul 17 01:47:02.578779 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:02.578565    2582 scope.go:117] "RemoveContainer" containerID="1b3ef908af70483685199fe2c8293b699b088a9fd748b78e814a2ede48dfafb3"
Jul 17 01:47:02.578779 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:47:02.578772    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt_odh-ai-gateway-infra(6d7de43a-7910-485f-bc1f-29abeb32afae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae"
Jul 17 01:47:03.584697 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:03.584670    2582 scope.go:117] "RemoveContainer" containerID="1b3ef908af70483685199fe2c8293b699b088a9fd748b78e814a2ede48dfafb3"
Jul 17 01:47:03.585089 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:47:03.584863    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt_odh-ai-gateway-infra(6d7de43a-7910-485f-bc1f-29abeb32afae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae"
Jul 17 01:47:07.590282 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.590243    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"]
Jul 17 01:47:07.590705 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.590657    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:47:07.590705 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.590670    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:47:07.590705 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.590681    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:47:07.590705 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.590687    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="595c0a1b-5895-4cb3-ba9c-fab0103bb682" containerName="revoke-keys"
Jul 17 01:47:07.590841 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.590764    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc6de7ba-95ff-4229-a327-a11be3de920a" containerName="revoke-keys"
Jul 17 01:47:07.595135 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.595118    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"
Jul 17 01:47:07.599453 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.599423    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"]
Jul 17 01:47:07.677045 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.677008    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab8d719-428f-4897-a206-d4978b1c95fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb\" (UID: \"aab8d719-428f-4897-a206-d4978b1c95fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"
Jul 17 01:47:07.778124 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.778085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab8d719-428f-4897-a206-d4978b1c95fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb\" (UID: \"aab8d719-428f-4897-a206-d4978b1c95fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"
Jul 17 01:47:07.778756 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.778737    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab8d719-428f-4897-a206-d4978b1c95fb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb\" (UID: \"aab8d719-428f-4897-a206-d4978b1c95fb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"
Jul 17 01:47:07.909276 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:07.909176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"
Jul 17 01:47:08.256449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:08.256423    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"]
Jul 17 01:47:08.258463 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:47:08.258435    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab8d719_428f_4897_a206_d4978b1c95fb.slice/crio-f2e7abd48298fc90a4766907b651ff61a4b854be282d6cb90aeb7782520794a5 WatchSource:0}: Error finding container f2e7abd48298fc90a4766907b651ff61a4b854be282d6cb90aeb7782520794a5: Status 404 returned error can't find the container with id f2e7abd48298fc90a4766907b651ff61a4b854be282d6cb90aeb7782520794a5
Jul 17 01:47:08.612060 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:08.612023    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" event={"ID":"aab8d719-428f-4897-a206-d4978b1c95fb","Type":"ContainerStarted","Data":"e23129a462172309e097a68930fca47b64673a470b4c5b29df83eb49a2011a90"}
Jul 17 01:47:08.612582 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:08.612085    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" event={"ID":"aab8d719-428f-4897-a206-d4978b1c95fb","Type":"ContainerStarted","Data":"f2e7abd48298fc90a4766907b651ff61a4b854be282d6cb90aeb7782520794a5"}
Jul 17 01:47:08.625465 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:08.625417    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" podStartSLOduration=1.625401832 podStartE2EDuration="1.625401832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:47:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:47:08.624556105 +0000 UTC m=+2036.128241967" watchObservedRunningTime="2026-07-17 01:47:08.625401832 +0000 UTC m=+2036.129087694"
Jul 17 01:47:09.618377 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:09.618345    2582 generic.go:356] "Generic (PLEG): container finished" podID="aab8d719-428f-4897-a206-d4978b1c95fb" containerID="e23129a462172309e097a68930fca47b64673a470b4c5b29df83eb49a2011a90" exitCode=6
Jul 17 01:47:09.618855 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:09.618445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" event={"ID":"aab8d719-428f-4897-a206-d4978b1c95fb","Type":"ContainerDied","Data":"e23129a462172309e097a68930fca47b64673a470b4c5b29df83eb49a2011a90"}
Jul 17 01:47:09.618855 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:09.618630    2582 scope.go:117] "RemoveContainer" containerID="e23129a462172309e097a68930fca47b64673a470b4c5b29df83eb49a2011a90"
Jul 17 01:47:10.625536 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:10.625427    2582 generic.go:356] "Generic (PLEG): container finished" podID="aab8d719-428f-4897-a206-d4978b1c95fb" containerID="a43d91a618d192caa9c412f6caa51d4cf7a306902f2757d84275e04d27825748" exitCode=6
Jul 17 01:47:10.625536 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:10.625518    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" event={"ID":"aab8d719-428f-4897-a206-d4978b1c95fb","Type":"ContainerDied","Data":"a43d91a618d192caa9c412f6caa51d4cf7a306902f2757d84275e04d27825748"}
Jul 17 01:47:10.626076 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:10.625574    2582 scope.go:117] "RemoveContainer" containerID="e23129a462172309e097a68930fca47b64673a470b4c5b29df83eb49a2011a90"
Jul 17 01:47:10.626076 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:10.625799    2582 scope.go:117] "RemoveContainer" containerID="a43d91a618d192caa9c412f6caa51d4cf7a306902f2757d84275e04d27825748"
Jul 17 01:47:10.626076 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:47:10.626035    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb_odh-ai-gateway-infra(aab8d719-428f-4897-a206-d4978b1c95fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" podUID="aab8d719-428f-4897-a206-d4978b1c95fb"
Jul 17 01:47:11.631880 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:11.631853    2582 scope.go:117] "RemoveContainer" containerID="a43d91a618d192caa9c412f6caa51d4cf7a306902f2757d84275e04d27825748"
Jul 17 01:47:11.632270 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:47:11.632062    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb_odh-ai-gateway-infra(aab8d719-428f-4897-a206-d4978b1c95fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" podUID="aab8d719-428f-4897-a206-d4978b1c95fb"
Jul 17 01:47:17.984726 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:17.984695    2582 scope.go:117] "RemoveContainer" containerID="1b3ef908af70483685199fe2c8293b699b088a9fd748b78e814a2ede48dfafb3"
Jul 17 01:47:18.667612 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:18.667581    2582 generic.go:356] "Generic (PLEG): container finished" podID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerID="d18b076214a8b879876c91d35e611bcd2b2a31e6eea7a6698add3a4c9219233a" exitCode=6
Jul 17 01:47:18.667753 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:18.667644    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" event={"ID":"6d7de43a-7910-485f-bc1f-29abeb32afae","Type":"ContainerDied","Data":"d18b076214a8b879876c91d35e611bcd2b2a31e6eea7a6698add3a4c9219233a"}
Jul 17 01:47:18.667753 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:18.667724    2582 scope.go:117] "RemoveContainer" containerID="1b3ef908af70483685199fe2c8293b699b088a9fd748b78e814a2ede48dfafb3"
Jul 17 01:47:18.668068 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:18.668050    2582 scope.go:117] "RemoveContainer" containerID="d18b076214a8b879876c91d35e611bcd2b2a31e6eea7a6698add3a4c9219233a"
Jul 17 01:47:18.668329 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:47:18.668305    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt_odh-ai-gateway-infra(6d7de43a-7910-485f-bc1f-29abeb32afae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae"
Jul 17 01:47:19.725458 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:19.725425    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"]
Jul 17 01:47:19.891268 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:19.891247    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"
Jul 17 01:47:19.986747 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:19.986646    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7de43a-7910-485f-bc1f-29abeb32afae-maas-api-service-ca\") pod \"6d7de43a-7910-485f-bc1f-29abeb32afae\" (UID: \"6d7de43a-7910-485f-bc1f-29abeb32afae\") "
Jul 17 01:47:19.987078 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:19.987042    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d7de43a-7910-485f-bc1f-29abeb32afae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6d7de43a-7910-485f-bc1f-29abeb32afae" (UID: "6d7de43a-7910-485f-bc1f-29abeb32afae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:47:20.088095 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:20.088054    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7de43a-7910-485f-bc1f-29abeb32afae-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:47:20.680165 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:20.680131    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt" event={"ID":"6d7de43a-7910-485f-bc1f-29abeb32afae","Type":"ContainerDied","Data":"c783488e5b42c3a203decb044faae2888185eb24d06b6bad201ecac48fbe2f61"}
Jul 17 01:47:20.680165 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:20.680157    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"
Jul 17 01:47:20.680441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:20.680188    2582 scope.go:117] "RemoveContainer" containerID="d18b076214a8b879876c91d35e611bcd2b2a31e6eea7a6698add3a4c9219233a"
Jul 17 01:47:20.700243 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:20.700221    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"]
Jul 17 01:47:20.705214 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:20.705192    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-c4bvt"]
Jul 17 01:47:20.990684 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:20.990602    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" path="/var/lib/kubelet/pods/6d7de43a-7910-485f-bc1f-29abeb32afae/volumes"
Jul 17 01:47:26.984812 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:26.984785    2582 scope.go:117] "RemoveContainer" containerID="a43d91a618d192caa9c412f6caa51d4cf7a306902f2757d84275e04d27825748"
Jul 17 01:47:27.714266 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:27.714227    2582 generic.go:356] "Generic (PLEG): container finished" podID="aab8d719-428f-4897-a206-d4978b1c95fb" containerID="9cf7f2d1d822f9c0ffe7d94195b59c8e24ad0eb39aee10f74b64bfd8c3388328" exitCode=6
Jul 17 01:47:27.714440 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:27.714302    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" event={"ID":"aab8d719-428f-4897-a206-d4978b1c95fb","Type":"ContainerDied","Data":"9cf7f2d1d822f9c0ffe7d94195b59c8e24ad0eb39aee10f74b64bfd8c3388328"}
Jul 17 01:47:27.714440 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:27.714365    2582 scope.go:117] "RemoveContainer" containerID="a43d91a618d192caa9c412f6caa51d4cf7a306902f2757d84275e04d27825748"
Jul 17 01:47:27.714729 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:27.714712    2582 scope.go:117] "RemoveContainer" containerID="9cf7f2d1d822f9c0ffe7d94195b59c8e24ad0eb39aee10f74b64bfd8c3388328"
Jul 17 01:47:27.714921 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:47:27.714897    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb_odh-ai-gateway-infra(aab8d719-428f-4897-a206-d4978b1c95fb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" podUID="aab8d719-428f-4897-a206-d4978b1c95fb"
Jul 17 01:47:28.738177 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:28.738142    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"]
Jul 17 01:47:28.877601 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:28.877572    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"
Jul 17 01:47:28.962175 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:28.962135    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab8d719-428f-4897-a206-d4978b1c95fb-maas-api-service-ca\") pod \"aab8d719-428f-4897-a206-d4978b1c95fb\" (UID: \"aab8d719-428f-4897-a206-d4978b1c95fb\") "
Jul 17 01:47:28.962561 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:28.962537    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aab8d719-428f-4897-a206-d4978b1c95fb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aab8d719-428f-4897-a206-d4978b1c95fb" (UID: "aab8d719-428f-4897-a206-d4978b1c95fb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:47:29.063322 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:29.063288    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab8d719-428f-4897-a206-d4978b1c95fb-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:47:29.727125 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:29.727093    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"
Jul 17 01:47:29.727125 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:29.727113    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb" event={"ID":"aab8d719-428f-4897-a206-d4978b1c95fb","Type":"ContainerDied","Data":"f2e7abd48298fc90a4766907b651ff61a4b854be282d6cb90aeb7782520794a5"}
Jul 17 01:47:29.727376 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:29.727194    2582 scope.go:117] "RemoveContainer" containerID="9cf7f2d1d822f9c0ffe7d94195b59c8e24ad0eb39aee10f74b64bfd8c3388328"
Jul 17 01:47:29.742105 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:29.742077    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"]
Jul 17 01:47:29.744584 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:29.744564    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-vzsfb"]
Jul 17 01:47:30.989834 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:30.989800    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" path="/var/lib/kubelet/pods/aab8d719-428f-4897-a206-d4978b1c95fb/volumes"
Jul 17 01:47:59.608568 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608473    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"]
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608838    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608850    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608860    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608866    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608872    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608878    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608882    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608889    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608913    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:47:59.608954 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608921    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:47:59.609263 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608985    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:47:59.609263 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608993    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:47:59.609263 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.608999    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:47:59.609263 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.609006    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:47:59.609263 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.609012    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:47:59.611703 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.611688    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"
Jul 17 01:47:59.614041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.614014    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:47:59.614185 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.614012    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:47:59.617659 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.617636    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"]
Jul 17 01:47:59.625478 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.625448    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80be099-28c5-4fe1-a094-61ae9d598ba1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt\" (UID: \"e80be099-28c5-4fe1-a094-61ae9d598ba1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"
Jul 17 01:47:59.726946 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.726914    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80be099-28c5-4fe1-a094-61ae9d598ba1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt\" (UID: \"e80be099-28c5-4fe1-a094-61ae9d598ba1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"
Jul 17 01:47:59.727586 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.727569    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80be099-28c5-4fe1-a094-61ae9d598ba1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt\" (UID: \"e80be099-28c5-4fe1-a094-61ae9d598ba1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"
Jul 17 01:47:59.923675 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:47:59.923588    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"
Jul 17 01:48:00.058468 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:00.058427    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"]
Jul 17 01:48:00.061973 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:48:00.061943    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80be099_28c5_4fe1_a094_61ae9d598ba1.slice/crio-74a6a8d52cebc2d1653c433b1a5b044e7325e4af5e304a7ab3e4330f46451b61 WatchSource:0}: Error finding container 74a6a8d52cebc2d1653c433b1a5b044e7325e4af5e304a7ab3e4330f46451b61: Status 404 returned error can't find the container with id 74a6a8d52cebc2d1653c433b1a5b044e7325e4af5e304a7ab3e4330f46451b61
Jul 17 01:48:00.884221 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:00.884188    2582 generic.go:356] "Generic (PLEG): container finished" podID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerID="53ad9478f633edc1094772c6e1686b0a001a93b5c290dc73b7812591979d7e26" exitCode=6
Jul 17 01:48:00.884620 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:00.884236    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt" event={"ID":"e80be099-28c5-4fe1-a094-61ae9d598ba1","Type":"ContainerDied","Data":"53ad9478f633edc1094772c6e1686b0a001a93b5c290dc73b7812591979d7e26"}
Jul 17 01:48:00.884620 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:00.884289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt" event={"ID":"e80be099-28c5-4fe1-a094-61ae9d598ba1","Type":"ContainerStarted","Data":"74a6a8d52cebc2d1653c433b1a5b044e7325e4af5e304a7ab3e4330f46451b61"}
Jul 17 01:48:00.884620 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:00.884476    2582 scope.go:117] "RemoveContainer" containerID="53ad9478f633edc1094772c6e1686b0a001a93b5c290dc73b7812591979d7e26"
Jul 17 01:48:01.891327 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:01.891235    2582 generic.go:356] "Generic (PLEG): container finished" podID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerID="7d0a8be1bda8f90363bedd1415902b784ad5eeccf9555285580c3ddb20588819" exitCode=6
Jul 17 01:48:01.891891 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:01.891328    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt" event={"ID":"e80be099-28c5-4fe1-a094-61ae9d598ba1","Type":"ContainerDied","Data":"7d0a8be1bda8f90363bedd1415902b784ad5eeccf9555285580c3ddb20588819"}
Jul 17 01:48:01.891891 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:01.891380    2582 scope.go:117] "RemoveContainer" containerID="53ad9478f633edc1094772c6e1686b0a001a93b5c290dc73b7812591979d7e26"
Jul 17 01:48:01.891891 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:01.891535    2582 scope.go:117] "RemoveContainer" containerID="7d0a8be1bda8f90363bedd1415902b784ad5eeccf9555285580c3ddb20588819"
Jul 17 01:48:01.891891 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:48:01.891797    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt_odh-ai-gateway-infra(e80be099-28c5-4fe1-a094-61ae9d598ba1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1"
Jul 17 01:48:02.899084 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:02.899051    2582 scope.go:117] "RemoveContainer" containerID="7d0a8be1bda8f90363bedd1415902b784ad5eeccf9555285580c3ddb20588819"
Jul 17 01:48:02.899482 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:48:02.899252    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt_odh-ai-gateway-infra(e80be099-28c5-4fe1-a094-61ae9d598ba1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1"
Jul 17 01:48:05.708707 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.708672    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"]
Jul 17 01:48:05.709146 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.709083    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:48:05.709146 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.709095    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d7de43a-7910-485f-bc1f-29abeb32afae" containerName="revoke-keys"
Jul 17 01:48:05.709238 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.709159    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="aab8d719-428f-4897-a206-d4978b1c95fb" containerName="revoke-keys"
Jul 17 01:48:05.713300 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.713279    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"
Jul 17 01:48:05.719273 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.719236    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"]
Jul 17 01:48:05.779011 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.778977    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca017b9-87f4-4e78-a998-3276618b955c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w\" (UID: \"3ca017b9-87f4-4e78-a998-3276618b955c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"
Jul 17 01:48:05.879822 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.879788    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca017b9-87f4-4e78-a998-3276618b955c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w\" (UID: \"3ca017b9-87f4-4e78-a998-3276618b955c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"
Jul 17 01:48:05.880427 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:05.880404    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca017b9-87f4-4e78-a998-3276618b955c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w\" (UID: \"3ca017b9-87f4-4e78-a998-3276618b955c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"
Jul 17 01:48:06.026929 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:06.026821    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"
Jul 17 01:48:06.163968 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:06.163934    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"]
Jul 17 01:48:06.166518 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:48:06.166474    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca017b9_87f4_4e78_a998_3276618b955c.slice/crio-9deb7017a3611daee899cc3a67e1909eef732473bc7d5616843070edb4c64a67 WatchSource:0}: Error finding container 9deb7017a3611daee899cc3a67e1909eef732473bc7d5616843070edb4c64a67: Status 404 returned error can't find the container with id 9deb7017a3611daee899cc3a67e1909eef732473bc7d5616843070edb4c64a67
Jul 17 01:48:06.918710 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:06.918680    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ca017b9-87f4-4e78-a998-3276618b955c" containerID="3e65c65158ad4f73cf475af201b3474ec66c0b7a89ea57013d009887bd9121cf" exitCode=7
Jul 17 01:48:06.919119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:06.918767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w" event={"ID":"3ca017b9-87f4-4e78-a998-3276618b955c","Type":"ContainerDied","Data":"3e65c65158ad4f73cf475af201b3474ec66c0b7a89ea57013d009887bd9121cf"}
Jul 17 01:48:06.919119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:06.918829    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w" event={"ID":"3ca017b9-87f4-4e78-a998-3276618b955c","Type":"ContainerStarted","Data":"9deb7017a3611daee899cc3a67e1909eef732473bc7d5616843070edb4c64a67"}
Jul 17 01:48:06.919119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:06.918922    2582 scope.go:117] "RemoveContainer" containerID="3e65c65158ad4f73cf475af201b3474ec66c0b7a89ea57013d009887bd9121cf"
Jul 17 01:48:07.925232 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:07.925143    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ca017b9-87f4-4e78-a998-3276618b955c" containerID="c8d86d7207f3d4e3e54af6b759b21fbc0baf4f265c4baabf0875208cbf0ed950" exitCode=7
Jul 17 01:48:07.925232 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:07.925219    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w" event={"ID":"3ca017b9-87f4-4e78-a998-3276618b955c","Type":"ContainerDied","Data":"c8d86d7207f3d4e3e54af6b759b21fbc0baf4f265c4baabf0875208cbf0ed950"}
Jul 17 01:48:07.925731 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:07.925271    2582 scope.go:117] "RemoveContainer" containerID="3e65c65158ad4f73cf475af201b3474ec66c0b7a89ea57013d009887bd9121cf"
Jul 17 01:48:07.925731 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:07.925468    2582 scope.go:117] "RemoveContainer" containerID="c8d86d7207f3d4e3e54af6b759b21fbc0baf4f265c4baabf0875208cbf0ed950"
Jul 17 01:48:07.925833 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:48:07.925738    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w_odh-ai-gateway-infra(3ca017b9-87f4-4e78-a998-3276618b955c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w" podUID="3ca017b9-87f4-4e78-a998-3276618b955c"
Jul 17 01:48:08.931912 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:08.931885    2582 scope.go:117] "RemoveContainer" containerID="c8d86d7207f3d4e3e54af6b759b21fbc0baf4f265c4baabf0875208cbf0ed950"
Jul 17 01:48:08.932337 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:48:08.932083    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w_odh-ai-gateway-infra(3ca017b9-87f4-4e78-a998-3276618b955c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w" podUID="3ca017b9-87f4-4e78-a998-3276618b955c"
Jul 17 01:48:14.326298 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:14.326182    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:48:14.351514 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:14.329435    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:48:14.351514 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:14.339425    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:48:14.351514 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:14.343062    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:48:17.984727 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:17.984702    2582 scope.go:117] "RemoveContainer" containerID="7d0a8be1bda8f90363bedd1415902b784ad5eeccf9555285580c3ddb20588819"
Jul 17 01:48:18.979349 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:18.979313    2582 generic.go:356] "Generic (PLEG): container finished" podID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerID="7be1a60e03dcfb488a61b3c50d4def504d3f6f40b1d51cfafb7dcdd26ba32b96" exitCode=6
Jul 17 01:48:18.979349 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:18.979351    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt" event={"ID":"e80be099-28c5-4fe1-a094-61ae9d598ba1","Type":"ContainerDied","Data":"7be1a60e03dcfb488a61b3c50d4def504d3f6f40b1d51cfafb7dcdd26ba32b96"}
Jul 17 01:48:18.979583 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:18.979408    2582 scope.go:117] "RemoveContainer" containerID="7d0a8be1bda8f90363bedd1415902b784ad5eeccf9555285580c3ddb20588819"
Jul 17 01:48:18.979759 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:18.979740    2582 scope.go:117] "RemoveContainer" containerID="7be1a60e03dcfb488a61b3c50d4def504d3f6f40b1d51cfafb7dcdd26ba32b96"
Jul 17 01:48:18.979986 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:48:18.979964    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt_odh-ai-gateway-infra(e80be099-28c5-4fe1-a094-61ae9d598ba1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1"
Jul 17 01:48:20.003768 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.003732    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"]
Jul 17 01:48:20.142421 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.142385    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"
Jul 17 01:48:20.201878 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.201845    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80be099-28c5-4fe1-a094-61ae9d598ba1-maas-api-service-ca\") pod \"e80be099-28c5-4fe1-a094-61ae9d598ba1\" (UID: \"e80be099-28c5-4fe1-a094-61ae9d598ba1\") "
Jul 17 01:48:20.202196 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.202174    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e80be099-28c5-4fe1-a094-61ae9d598ba1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e80be099-28c5-4fe1-a094-61ae9d598ba1" (UID: "e80be099-28c5-4fe1-a094-61ae9d598ba1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:48:20.303179 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.303092    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80be099-28c5-4fe1-a094-61ae9d598ba1-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:48:20.984986 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.984951    2582 scope.go:117] "RemoveContainer" containerID="c8d86d7207f3d4e3e54af6b759b21fbc0baf4f265c4baabf0875208cbf0ed950"
Jul 17 01:48:20.994816 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.994790    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt" event={"ID":"e80be099-28c5-4fe1-a094-61ae9d598ba1","Type":"ContainerDied","Data":"74a6a8d52cebc2d1653c433b1a5b044e7325e4af5e304a7ab3e4330f46451b61"}
Jul 17 01:48:20.994816 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.994815    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"
Jul 17 01:48:20.994997 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:20.994837    2582 scope.go:117] "RemoveContainer" containerID="7be1a60e03dcfb488a61b3c50d4def504d3f6f40b1d51cfafb7dcdd26ba32b96"
Jul 17 01:48:21.033822 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:21.033798    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"]
Jul 17 01:48:21.039235 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:21.039213    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-chlbt"]
Jul 17 01:48:22.000913 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:22.000829    2582 generic.go:356] "Generic (PLEG): container finished" podID="3ca017b9-87f4-4e78-a998-3276618b955c" containerID="4a3f2fd6b0df3d079a9b929fcf461f1fa250ef5d693cb1c07f41b6713f64125f" exitCode=7
Jul 17 01:48:22.000913 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:22.000896    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w" event={"ID":"3ca017b9-87f4-4e78-a998-3276618b955c","Type":"ContainerDied","Data":"4a3f2fd6b0df3d079a9b929fcf461f1fa250ef5d693cb1c07f41b6713f64125f"}
Jul 17 01:48:22.001164 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:22.000956    2582 scope.go:117] "RemoveContainer" containerID="c8d86d7207f3d4e3e54af6b759b21fbc0baf4f265c4baabf0875208cbf0ed950"
Jul 17 01:48:22.001322 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:22.001297    2582 scope.go:117] "RemoveContainer" containerID="4a3f2fd6b0df3d079a9b929fcf461f1fa250ef5d693cb1c07f41b6713f64125f"
Jul 17 01:48:22.001586 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:48:22.001565    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w_odh-ai-gateway-infra(3ca017b9-87f4-4e78-a998-3276618b955c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w" podUID="3ca017b9-87f4-4e78-a998-3276618b955c"
Jul 17 01:48:22.990868 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:22.990831    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" path="/var/lib/kubelet/pods/e80be099-28c5-4fe1-a094-61ae9d598ba1/volumes"
Jul 17 01:48:23.024932 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:23.024902    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"]
Jul 17 01:48:23.164306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:23.164282    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"
Jul 17 01:48:23.229311 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:23.229223    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca017b9-87f4-4e78-a998-3276618b955c-maas-api-service-ca\") pod \"3ca017b9-87f4-4e78-a998-3276618b955c\" (UID: \"3ca017b9-87f4-4e78-a998-3276618b955c\") "
Jul 17 01:48:23.229602 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:23.229581    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ca017b9-87f4-4e78-a998-3276618b955c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3ca017b9-87f4-4e78-a998-3276618b955c" (UID: "3ca017b9-87f4-4e78-a998-3276618b955c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:48:23.330074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:23.330040    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ca017b9-87f4-4e78-a998-3276618b955c-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:48:24.013773 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:24.013740    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"
Jul 17 01:48:24.013773 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:24.013760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w" event={"ID":"3ca017b9-87f4-4e78-a998-3276618b955c","Type":"ContainerDied","Data":"9deb7017a3611daee899cc3a67e1909eef732473bc7d5616843070edb4c64a67"}
Jul 17 01:48:24.014311 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:24.013817    2582 scope.go:117] "RemoveContainer" containerID="4a3f2fd6b0df3d079a9b929fcf461f1fa250ef5d693cb1c07f41b6713f64125f"
Jul 17 01:48:24.035308 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:24.035276    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"]
Jul 17 01:48:24.043052 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:24.043029    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-n6v6w"]
Jul 17 01:48:24.989951 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:24.989916    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" path="/var/lib/kubelet/pods/3ca017b9-87f4-4e78-a998-3276618b955c/volumes"
Jul 17 01:48:51.008440 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:51.008370    2582 pod_container_manager_linux.go:217] "Failed to delete cgroup paths" cgroupName=["kubepods","burstable","pode80be099-28c5-4fe1-a094-61ae9d598ba1"] err="unable to destroy cgroup paths for cgroup [kubepods burstable pode80be099-28c5-4fe1-a094-61ae9d598ba1] : Timed out while waiting for systemd to remove kubepods-burstable-pode80be099_28c5_4fe1_a094_61ae9d598ba1.slice"
Jul 17 01:48:55.738925 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.738891    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"]
Jul 17 01:48:55.739296 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739256    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:48:55.739296 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739268    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:48:55.739296 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739282    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:48:55.739296 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739288    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739302    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739308    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739316    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739321    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739329    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739334    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739341    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739346    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739396    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739403    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739409    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739415    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.739454 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.739421    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e80be099-28c5-4fe1-a094-61ae9d598ba1" containerName="revoke-keys"
Jul 17 01:48:55.742354 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.742338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"
Jul 17 01:48:55.745119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.745098    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:48:55.745269 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.745131    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:48:55.748622 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.748598    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"]
Jul 17 01:48:55.817810 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.817775    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q\" (UID: \"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"
Jul 17 01:48:55.919147 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.919108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q\" (UID: \"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"
Jul 17 01:48:55.919768 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:55.919747    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q\" (UID: \"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"
Jul 17 01:48:56.055177 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:56.055075    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"
Jul 17 01:48:56.190041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:56.190014    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"]
Jul 17 01:48:56.191518 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:48:56.191469    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6ea6cf_0ce0_4224_8fe0_a83eb2ee75b3.slice/crio-14e2f4144e9d4732399b97d1afd05fb4747a49e8c5571c0c05f2409beea2163e WatchSource:0}: Error finding container 14e2f4144e9d4732399b97d1afd05fb4747a49e8c5571c0c05f2409beea2163e: Status 404 returned error can't find the container with id 14e2f4144e9d4732399b97d1afd05fb4747a49e8c5571c0c05f2409beea2163e
Jul 17 01:48:57.168813 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:57.168711    2582 generic.go:356] "Generic (PLEG): container finished" podID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerID="b8d4837028a28f12bfdb13e9ca8db157a8bcfaa843514eb7fec7692c66eb0adb" exitCode=7
Jul 17 01:48:57.169229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:57.168796    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" event={"ID":"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3","Type":"ContainerDied","Data":"b8d4837028a28f12bfdb13e9ca8db157a8bcfaa843514eb7fec7692c66eb0adb"}
Jul 17 01:48:57.169229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:57.168859    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" event={"ID":"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3","Type":"ContainerStarted","Data":"14e2f4144e9d4732399b97d1afd05fb4747a49e8c5571c0c05f2409beea2163e"}
Jul 17 01:48:57.169229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:57.168990    2582 scope.go:117] "RemoveContainer" containerID="b8d4837028a28f12bfdb13e9ca8db157a8bcfaa843514eb7fec7692c66eb0adb"
Jul 17 01:48:58.175932 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:58.175898    2582 generic.go:356] "Generic (PLEG): container finished" podID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerID="a94a16a8f1702671b153145518932c4790f10fbea878c48e275e78efe5b75049" exitCode=7
Jul 17 01:48:58.176384 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:58.175983    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" event={"ID":"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3","Type":"ContainerDied","Data":"a94a16a8f1702671b153145518932c4790f10fbea878c48e275e78efe5b75049"}
Jul 17 01:48:58.176384 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:58.176038    2582 scope.go:117] "RemoveContainer" containerID="b8d4837028a28f12bfdb13e9ca8db157a8bcfaa843514eb7fec7692c66eb0adb"
Jul 17 01:48:58.176384 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:58.176303    2582 scope.go:117] "RemoveContainer" containerID="a94a16a8f1702671b153145518932c4790f10fbea878c48e275e78efe5b75049"
Jul 17 01:48:58.176599 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:48:58.176523    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q_odh-ai-gateway-infra(cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3"
Jul 17 01:48:59.182356 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:48:59.182331    2582 scope.go:117] "RemoveContainer" containerID="a94a16a8f1702671b153145518932c4790f10fbea878c48e275e78efe5b75049"
Jul 17 01:48:59.182775 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:48:59.182532    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q_odh-ai-gateway-infra(cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3"
Jul 17 01:49:13.985242 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:13.985211    2582 scope.go:117] "RemoveContainer" containerID="a94a16a8f1702671b153145518932c4790f10fbea878c48e275e78efe5b75049"
Jul 17 01:49:14.251569 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:14.251509    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" event={"ID":"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3","Type":"ContainerStarted","Data":"54c2ed737d27d3df818e9874791a71dd9d6f0bfd02d2de6e8462dad767f0b15b"}
Jul 17 01:49:14.272064 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:14.272015    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" podStartSLOduration=19.272001205 podStartE2EDuration="19.272001205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:49:14.270664556 +0000 UTC m=+2161.774350421" watchObservedRunningTime="2026-07-17 01:49:14.272001205 +0000 UTC m=+2161.775687066"
Jul 17 01:49:15.257959 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:15.257920    2582 generic.go:356] "Generic (PLEG): container finished" podID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerID="54c2ed737d27d3df818e9874791a71dd9d6f0bfd02d2de6e8462dad767f0b15b" exitCode=7
Jul 17 01:49:15.258397 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:15.257988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" event={"ID":"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3","Type":"ContainerDied","Data":"54c2ed737d27d3df818e9874791a71dd9d6f0bfd02d2de6e8462dad767f0b15b"}
Jul 17 01:49:15.258397 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:15.258064    2582 scope.go:117] "RemoveContainer" containerID="a94a16a8f1702671b153145518932c4790f10fbea878c48e275e78efe5b75049"
Jul 17 01:49:15.258529 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:15.258401    2582 scope.go:117] "RemoveContainer" containerID="54c2ed737d27d3df818e9874791a71dd9d6f0bfd02d2de6e8462dad767f0b15b"
Jul 17 01:49:15.258680 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:49:15.258657    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q_odh-ai-gateway-infra(cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3"
Jul 17 01:49:15.280299 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:15.280269    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"]
Jul 17 01:49:16.396516 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:16.396479    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"
Jul 17 01:49:16.496647 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:16.496615    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3-maas-api-service-ca\") pod \"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3\" (UID: \"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3\") "
Jul 17 01:49:16.497008 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:16.496986    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" (UID: "cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:49:16.598279 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:16.598190    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:49:17.272617 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:17.272535    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q" event={"ID":"cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3","Type":"ContainerDied","Data":"14e2f4144e9d4732399b97d1afd05fb4747a49e8c5571c0c05f2409beea2163e"}
Jul 17 01:49:17.272617 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:17.272570    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"
Jul 17 01:49:17.272617 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:17.272595    2582 scope.go:117] "RemoveContainer" containerID="54c2ed737d27d3df818e9874791a71dd9d6f0bfd02d2de6e8462dad767f0b15b"
Jul 17 01:49:17.287573 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:17.287547    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"]
Jul 17 01:49:17.293153 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:17.293131    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-k4j2q"]
Jul 17 01:49:18.994523 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:18.993384    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" path="/var/lib/kubelet/pods/cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3/volumes"
Jul 17 01:49:47.721033 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.720996    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"]
Jul 17 01:49:47.721455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721380    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721391    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721407    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721412    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721418    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721455 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721423    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721684 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721472    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721684 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721480    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721684 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721485    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf6ea6cf-0ce0-4224-8fe0-a83eb2ee75b3" containerName="revoke-keys"
Jul 17 01:49:47.721684 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.721511    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca017b9-87f4-4e78-a998-3276618b955c" containerName="revoke-keys"
Jul 17 01:49:47.724721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.724706    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"
Jul 17 01:49:47.727024 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.727009    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:49:47.727083 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.727065    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:49:47.730545 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.730516    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"]
Jul 17 01:49:47.779347 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.779319    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30c738d7-6336-40f0-932a-67fe6d49a908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq\" (UID: \"30c738d7-6336-40f0-932a-67fe6d49a908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"
Jul 17 01:49:47.880321 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.880285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30c738d7-6336-40f0-932a-67fe6d49a908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq\" (UID: \"30c738d7-6336-40f0-932a-67fe6d49a908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"
Jul 17 01:49:47.880915 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:47.880899    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30c738d7-6336-40f0-932a-67fe6d49a908-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq\" (UID: \"30c738d7-6336-40f0-932a-67fe6d49a908\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"
Jul 17 01:49:48.036671 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:48.036569    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"
Jul 17 01:49:48.173285 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:48.173262    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"]
Jul 17 01:49:48.175593 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:49:48.175565    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c738d7_6336_40f0_932a_67fe6d49a908.slice/crio-a0a0025773fec8b0f0cd03261036971e5c053460e802d9161a1eba570e5ccbb5 WatchSource:0}: Error finding container a0a0025773fec8b0f0cd03261036971e5c053460e802d9161a1eba570e5ccbb5: Status 404 returned error can't find the container with id a0a0025773fec8b0f0cd03261036971e5c053460e802d9161a1eba570e5ccbb5
Jul 17 01:49:48.420229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:48.420154    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" event={"ID":"30c738d7-6336-40f0-932a-67fe6d49a908","Type":"ContainerStarted","Data":"a0a0025773fec8b0f0cd03261036971e5c053460e802d9161a1eba570e5ccbb5"}
Jul 17 01:49:49.427224 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:49.427192    2582 generic.go:356] "Generic (PLEG): container finished" podID="30c738d7-6336-40f0-932a-67fe6d49a908" containerID="304c214d4c5fbf90c1ea0873ef2c46a2996bbec382fb895f2f76b7d7a87fceff" exitCode=6
Jul 17 01:49:49.427628 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:49.427280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" event={"ID":"30c738d7-6336-40f0-932a-67fe6d49a908","Type":"ContainerDied","Data":"304c214d4c5fbf90c1ea0873ef2c46a2996bbec382fb895f2f76b7d7a87fceff"}
Jul 17 01:49:49.427628 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:49.427434    2582 scope.go:117] "RemoveContainer" containerID="304c214d4c5fbf90c1ea0873ef2c46a2996bbec382fb895f2f76b7d7a87fceff"
Jul 17 01:49:50.433431 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:50.433397    2582 generic.go:356] "Generic (PLEG): container finished" podID="30c738d7-6336-40f0-932a-67fe6d49a908" containerID="fe5660e3832df4056804dce75cf8da285d18c43ec5aaeae6fbb2bb8e6468449c" exitCode=6
Jul 17 01:49:50.433887 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:50.433440    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" event={"ID":"30c738d7-6336-40f0-932a-67fe6d49a908","Type":"ContainerDied","Data":"fe5660e3832df4056804dce75cf8da285d18c43ec5aaeae6fbb2bb8e6468449c"}
Jul 17 01:49:50.433887 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:50.433523    2582 scope.go:117] "RemoveContainer" containerID="304c214d4c5fbf90c1ea0873ef2c46a2996bbec382fb895f2f76b7d7a87fceff"
Jul 17 01:49:50.433887 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:50.433698    2582 scope.go:117] "RemoveContainer" containerID="fe5660e3832df4056804dce75cf8da285d18c43ec5aaeae6fbb2bb8e6468449c"
Jul 17 01:49:50.434048 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:49:50.433915    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq_odh-ai-gateway-infra(30c738d7-6336-40f0-932a-67fe6d49a908)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" podUID="30c738d7-6336-40f0-932a-67fe6d49a908"
Jul 17 01:49:51.440252 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:51.440225    2582 scope.go:117] "RemoveContainer" containerID="fe5660e3832df4056804dce75cf8da285d18c43ec5aaeae6fbb2bb8e6468449c"
Jul 17 01:49:51.440673 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:49:51.440420    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq_odh-ai-gateway-infra(30c738d7-6336-40f0-932a-67fe6d49a908)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" podUID="30c738d7-6336-40f0-932a-67fe6d49a908"
Jul 17 01:49:59.055317 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:59.055280    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:49:59.695119 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:59.695072    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"]
Jul 17 01:49:59.699716 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:59.699699    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"
Jul 17 01:49:59.705267 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:59.705237    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"]
Jul 17 01:49:59.787368 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:59.787331    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d92193b1-d25e-4e57-8768-9001552a26f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg\" (UID: \"d92193b1-d25e-4e57-8768-9001552a26f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"
Jul 17 01:49:59.888035 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:59.887995    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d92193b1-d25e-4e57-8768-9001552a26f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg\" (UID: \"d92193b1-d25e-4e57-8768-9001552a26f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"
Jul 17 01:49:59.888664 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:49:59.888646    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d92193b1-d25e-4e57-8768-9001552a26f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg\" (UID: \"d92193b1-d25e-4e57-8768-9001552a26f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"
Jul 17 01:50:00.011564 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:00.011440    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"
Jul 17 01:50:00.145804 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:00.145771    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"]
Jul 17 01:50:00.483156 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:00.483116    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" event={"ID":"d92193b1-d25e-4e57-8768-9001552a26f1","Type":"ContainerStarted","Data":"9453818b08f339c639bf7490aac0beacd26ce32a310fe1c8feb207f9e696e85b"}
Jul 17 01:50:00.483328 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:00.483181    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" event={"ID":"d92193b1-d25e-4e57-8768-9001552a26f1","Type":"ContainerStarted","Data":"0e81200b056129f5b23f1c33b1da6f53c0cd96df41ee2015a44781d03762c6d2"}
Jul 17 01:50:00.497961 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:00.497909    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" podStartSLOduration=1.497892104 podStartE2EDuration="1.497892104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:50:00.496339139 +0000 UTC m=+2208.000025002" watchObservedRunningTime="2026-07-17 01:50:00.497892104 +0000 UTC m=+2208.001577965"
Jul 17 01:50:01.489690 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:01.489657    2582 generic.go:356] "Generic (PLEG): container finished" podID="d92193b1-d25e-4e57-8768-9001552a26f1" containerID="9453818b08f339c639bf7490aac0beacd26ce32a310fe1c8feb207f9e696e85b" exitCode=6
Jul 17 01:50:01.490108 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:01.489744    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" event={"ID":"d92193b1-d25e-4e57-8768-9001552a26f1","Type":"ContainerDied","Data":"9453818b08f339c639bf7490aac0beacd26ce32a310fe1c8feb207f9e696e85b"}
Jul 17 01:50:01.490108 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:01.489901    2582 scope.go:117] "RemoveContainer" containerID="9453818b08f339c639bf7490aac0beacd26ce32a310fe1c8feb207f9e696e85b"
Jul 17 01:50:02.496077 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:02.496037    2582 generic.go:356] "Generic (PLEG): container finished" podID="d92193b1-d25e-4e57-8768-9001552a26f1" containerID="537c764fe6a51cd21a77dc8fcf34f43280e1ae9f9565e88a9ae6e580db1dbfd4" exitCode=6
Jul 17 01:50:02.496560 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:02.496131    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" event={"ID":"d92193b1-d25e-4e57-8768-9001552a26f1","Type":"ContainerDied","Data":"537c764fe6a51cd21a77dc8fcf34f43280e1ae9f9565e88a9ae6e580db1dbfd4"}
Jul 17 01:50:02.496560 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:02.496242    2582 scope.go:117] "RemoveContainer" containerID="9453818b08f339c639bf7490aac0beacd26ce32a310fe1c8feb207f9e696e85b"
Jul 17 01:50:02.496560 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:02.496322    2582 scope.go:117] "RemoveContainer" containerID="537c764fe6a51cd21a77dc8fcf34f43280e1ae9f9565e88a9ae6e580db1dbfd4"
Jul 17 01:50:02.496560 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:02.496539    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg_odh-ai-gateway-infra(d92193b1-d25e-4e57-8768-9001552a26f1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" podUID="d92193b1-d25e-4e57-8768-9001552a26f1"
Jul 17 01:50:02.987525 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:02.987476    2582 scope.go:117] "RemoveContainer" containerID="fe5660e3832df4056804dce75cf8da285d18c43ec5aaeae6fbb2bb8e6468449c"
Jul 17 01:50:03.502829 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:03.502728    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" event={"ID":"30c738d7-6336-40f0-932a-67fe6d49a908","Type":"ContainerStarted","Data":"006990caacf66d3dc067f41a0cd17e47ea393ae1a2aa7c3c78b238c161647fea"}
Jul 17 01:50:03.504622 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:03.504602    2582 scope.go:117] "RemoveContainer" containerID="537c764fe6a51cd21a77dc8fcf34f43280e1ae9f9565e88a9ae6e580db1dbfd4"
Jul 17 01:50:03.504789 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:03.504764    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg_odh-ai-gateway-infra(d92193b1-d25e-4e57-8768-9001552a26f1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" podUID="d92193b1-d25e-4e57-8768-9001552a26f1"
Jul 17 01:50:03.518134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:03.518087    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" podStartSLOduration=16.518071264 podStartE2EDuration="16.518071264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:49:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:50:03.515680852 +0000 UTC m=+2211.019366714" watchObservedRunningTime="2026-07-17 01:50:03.518071264 +0000 UTC m=+2211.021757150"
Jul 17 01:50:04.511897 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:04.511866    2582 generic.go:356] "Generic (PLEG): container finished" podID="30c738d7-6336-40f0-932a-67fe6d49a908" containerID="006990caacf66d3dc067f41a0cd17e47ea393ae1a2aa7c3c78b238c161647fea" exitCode=6
Jul 17 01:50:04.512323 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:04.511909    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" event={"ID":"30c738d7-6336-40f0-932a-67fe6d49a908","Type":"ContainerDied","Data":"006990caacf66d3dc067f41a0cd17e47ea393ae1a2aa7c3c78b238c161647fea"}
Jul 17 01:50:04.512323 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:04.511956    2582 scope.go:117] "RemoveContainer" containerID="fe5660e3832df4056804dce75cf8da285d18c43ec5aaeae6fbb2bb8e6468449c"
Jul 17 01:50:04.512323 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:04.512283    2582 scope.go:117] "RemoveContainer" containerID="006990caacf66d3dc067f41a0cd17e47ea393ae1a2aa7c3c78b238c161647fea"
Jul 17 01:50:04.512522 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:04.512473    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq_odh-ai-gateway-infra(30c738d7-6336-40f0-932a-67fe6d49a908)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" podUID="30c738d7-6336-40f0-932a-67fe6d49a908"
Jul 17 01:50:04.525180 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:04.525154    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"]
Jul 17 01:50:05.652377 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:05.652353    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"
Jul 17 01:50:05.741885 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:05.741845    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30c738d7-6336-40f0-932a-67fe6d49a908-maas-api-service-ca\") pod \"30c738d7-6336-40f0-932a-67fe6d49a908\" (UID: \"30c738d7-6336-40f0-932a-67fe6d49a908\") "
Jul 17 01:50:05.742214 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:05.742190    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30c738d7-6336-40f0-932a-67fe6d49a908-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30c738d7-6336-40f0-932a-67fe6d49a908" (UID: "30c738d7-6336-40f0-932a-67fe6d49a908"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:50:05.843601 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:05.843485    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30c738d7-6336-40f0-932a-67fe6d49a908-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:50:06.526267 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:06.526238    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"
Jul 17 01:50:06.526452 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:06.526237    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq" event={"ID":"30c738d7-6336-40f0-932a-67fe6d49a908","Type":"ContainerDied","Data":"a0a0025773fec8b0f0cd03261036971e5c053460e802d9161a1eba570e5ccbb5"}
Jul 17 01:50:06.526452 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:06.526378    2582 scope.go:117] "RemoveContainer" containerID="006990caacf66d3dc067f41a0cd17e47ea393ae1a2aa7c3c78b238c161647fea"
Jul 17 01:50:06.548812 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:06.548785    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"]
Jul 17 01:50:06.556520 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:06.556475    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-zs2nq"]
Jul 17 01:50:06.991105 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:06.991055    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" path="/var/lib/kubelet/pods/30c738d7-6336-40f0-932a-67fe6d49a908/volumes"
Jul 17 01:50:11.458912 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:11.458873    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:50:14.990753 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:14.990722    2582 scope.go:117] "RemoveContainer" containerID="537c764fe6a51cd21a77dc8fcf34f43280e1ae9f9565e88a9ae6e580db1dbfd4"
Jul 17 01:50:15.570006 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:15.569973    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" event={"ID":"d92193b1-d25e-4e57-8768-9001552a26f1","Type":"ContainerStarted","Data":"c83cb5a8a2532546e3ad03ed040eacba8a5641cc1889e9d3f29d5b6dec574b98"}
Jul 17 01:50:16.576156 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:16.576122    2582 generic.go:356] "Generic (PLEG): container finished" podID="d92193b1-d25e-4e57-8768-9001552a26f1" containerID="c83cb5a8a2532546e3ad03ed040eacba8a5641cc1889e9d3f29d5b6dec574b98" exitCode=6
Jul 17 01:50:16.576599 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:16.576192    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" event={"ID":"d92193b1-d25e-4e57-8768-9001552a26f1","Type":"ContainerDied","Data":"c83cb5a8a2532546e3ad03ed040eacba8a5641cc1889e9d3f29d5b6dec574b98"}
Jul 17 01:50:16.576599 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:16.576259    2582 scope.go:117] "RemoveContainer" containerID="537c764fe6a51cd21a77dc8fcf34f43280e1ae9f9565e88a9ae6e580db1dbfd4"
Jul 17 01:50:16.576599 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:16.576514    2582 scope.go:117] "RemoveContainer" containerID="c83cb5a8a2532546e3ad03ed040eacba8a5641cc1889e9d3f29d5b6dec574b98"
Jul 17 01:50:16.576792 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:16.576749    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg_odh-ai-gateway-infra(d92193b1-d25e-4e57-8768-9001552a26f1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" podUID="d92193b1-d25e-4e57-8768-9001552a26f1"
Jul 17 01:50:16.596427 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:16.596403    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"]
Jul 17 01:50:17.715510 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:17.715468    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"
Jul 17 01:50:17.857875 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:17.857785    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d92193b1-d25e-4e57-8768-9001552a26f1-maas-api-service-ca\") pod \"d92193b1-d25e-4e57-8768-9001552a26f1\" (UID: \"d92193b1-d25e-4e57-8768-9001552a26f1\") "
Jul 17 01:50:17.858172 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:17.858147    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d92193b1-d25e-4e57-8768-9001552a26f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d92193b1-d25e-4e57-8768-9001552a26f1" (UID: "d92193b1-d25e-4e57-8768-9001552a26f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:50:17.958929 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:17.958889    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d92193b1-d25e-4e57-8768-9001552a26f1-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:50:18.587875 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:18.587845    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"
Jul 17 01:50:18.588145 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:18.587876    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg" event={"ID":"d92193b1-d25e-4e57-8768-9001552a26f1","Type":"ContainerDied","Data":"0e81200b056129f5b23f1c33b1da6f53c0cd96df41ee2015a44781d03762c6d2"}
Jul 17 01:50:18.588145 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:18.587934    2582 scope.go:117] "RemoveContainer" containerID="c83cb5a8a2532546e3ad03ed040eacba8a5641cc1889e9d3f29d5b6dec574b98"
Jul 17 01:50:18.608808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:18.608769    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"]
Jul 17 01:50:18.612553 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:18.612528    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-s9bxg"]
Jul 17 01:50:18.991035 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:18.991003    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" path="/var/lib/kubelet/pods/d92193b1-d25e-4e57-8768-9001552a26f1/volumes"
Jul 17 01:50:37.845187 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845103    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"]
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845514    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845529    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845548    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845554    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845566    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845572    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845583    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845589    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845642    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845651    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845657    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845662    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:37.845725 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.845668    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:37.850098 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.850081    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"
Jul 17 01:50:37.853393 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.853345    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:50:37.853554 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.853445    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:50:37.854788 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.854766    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"]
Jul 17 01:50:37.935567 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:37.935522    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/379f8806-8c0b-4c8e-bb62-03a89764ee77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh\" (UID: \"379f8806-8c0b-4c8e-bb62-03a89764ee77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"
Jul 17 01:50:38.037041 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:38.037003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/379f8806-8c0b-4c8e-bb62-03a89764ee77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh\" (UID: \"379f8806-8c0b-4c8e-bb62-03a89764ee77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"
Jul 17 01:50:38.037689 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:38.037667    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/379f8806-8c0b-4c8e-bb62-03a89764ee77-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh\" (UID: \"379f8806-8c0b-4c8e-bb62-03a89764ee77\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"
Jul 17 01:50:38.162582 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:38.162482    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"
Jul 17 01:50:38.293743 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:38.293721    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"]
Jul 17 01:50:38.295961 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:50:38.295932    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379f8806_8c0b_4c8e_bb62_03a89764ee77.slice/crio-67abc7aebfa19a0ed9cf065e24a43a06e5ab702f2c9bdc974b415b1c9c5c99dc WatchSource:0}: Error finding container 67abc7aebfa19a0ed9cf065e24a43a06e5ab702f2c9bdc974b415b1c9c5c99dc: Status 404 returned error can't find the container with id 67abc7aebfa19a0ed9cf065e24a43a06e5ab702f2c9bdc974b415b1c9c5c99dc
Jul 17 01:50:38.689720 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:38.689684    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" event={"ID":"379f8806-8c0b-4c8e-bb62-03a89764ee77","Type":"ContainerStarted","Data":"de792140aa66dbcdae77b69abc7ac9d1cfdeb3a2a9050117dc94f3c6d33c3055"}
Jul 17 01:50:38.689878 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:38.689753    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" event={"ID":"379f8806-8c0b-4c8e-bb62-03a89764ee77","Type":"ContainerStarted","Data":"67abc7aebfa19a0ed9cf065e24a43a06e5ab702f2c9bdc974b415b1c9c5c99dc"}
Jul 17 01:50:38.704361 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:38.704313    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" podStartSLOduration=1.704298209 podStartE2EDuration="1.704298209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:50:38.703426581 +0000 UTC m=+2246.207112454" watchObservedRunningTime="2026-07-17 01:50:38.704298209 +0000 UTC m=+2246.207984081"
Jul 17 01:50:39.696603 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:39.696569    2582 generic.go:356] "Generic (PLEG): container finished" podID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerID="de792140aa66dbcdae77b69abc7ac9d1cfdeb3a2a9050117dc94f3c6d33c3055" exitCode=6
Jul 17 01:50:39.697035 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:39.696660    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" event={"ID":"379f8806-8c0b-4c8e-bb62-03a89764ee77","Type":"ContainerDied","Data":"de792140aa66dbcdae77b69abc7ac9d1cfdeb3a2a9050117dc94f3c6d33c3055"}
Jul 17 01:50:39.697035 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:39.696833    2582 scope.go:117] "RemoveContainer" containerID="de792140aa66dbcdae77b69abc7ac9d1cfdeb3a2a9050117dc94f3c6d33c3055"
Jul 17 01:50:40.703786 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:40.703750    2582 generic.go:356] "Generic (PLEG): container finished" podID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerID="5216451122a05dd6c568d269fd8b64981961e229f436caaee7b0ed39fe3254c6" exitCode=6
Jul 17 01:50:40.704250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:40.703842    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" event={"ID":"379f8806-8c0b-4c8e-bb62-03a89764ee77","Type":"ContainerDied","Data":"5216451122a05dd6c568d269fd8b64981961e229f436caaee7b0ed39fe3254c6"}
Jul 17 01:50:40.704250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:40.703903    2582 scope.go:117] "RemoveContainer" containerID="de792140aa66dbcdae77b69abc7ac9d1cfdeb3a2a9050117dc94f3c6d33c3055"
Jul 17 01:50:40.704250 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:40.704172    2582 scope.go:117] "RemoveContainer" containerID="5216451122a05dd6c568d269fd8b64981961e229f436caaee7b0ed39fe3254c6"
Jul 17 01:50:40.704409 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:40.704384    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh_odh-ai-gateway-infra(379f8806-8c0b-4c8e-bb62-03a89764ee77)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77"
Jul 17 01:50:41.710134 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:41.710105    2582 scope.go:117] "RemoveContainer" containerID="5216451122a05dd6c568d269fd8b64981961e229f436caaee7b0ed39fe3254c6"
Jul 17 01:50:41.710519 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:41.710302    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh_odh-ai-gateway-infra(379f8806-8c0b-4c8e-bb62-03a89764ee77)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77"
Jul 17 01:50:54.984966 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:54.984928    2582 scope.go:117] "RemoveContainer" containerID="5216451122a05dd6c568d269fd8b64981961e229f436caaee7b0ed39fe3254c6"
Jul 17 01:50:55.780986 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.780954    2582 generic.go:356] "Generic (PLEG): container finished" podID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerID="8ffca540afadbca8851ddb468d494da67b02afeccee8ae861ab1b372558c1722" exitCode=6
Jul 17 01:50:55.781173 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.780993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" event={"ID":"379f8806-8c0b-4c8e-bb62-03a89764ee77","Type":"ContainerDied","Data":"8ffca540afadbca8851ddb468d494da67b02afeccee8ae861ab1b372558c1722"}
Jul 17 01:50:55.781173 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.781050    2582 scope.go:117] "RemoveContainer" containerID="5216451122a05dd6c568d269fd8b64981961e229f436caaee7b0ed39fe3254c6"
Jul 17 01:50:55.781340 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.781319    2582 scope.go:117] "RemoveContainer" containerID="8ffca540afadbca8851ddb468d494da67b02afeccee8ae861ab1b372558c1722"
Jul 17 01:50:55.781632 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:55.781603    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh_odh-ai-gateway-infra(379f8806-8c0b-4c8e-bb62-03a89764ee77)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77"
Jul 17 01:50:55.822212 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.822177    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"]
Jul 17 01:50:55.822553 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.822541    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:55.822602 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.822554    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:55.822602 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.822576    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:55.822602 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.822582    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d92193b1-d25e-4e57-8768-9001552a26f1" containerName="revoke-keys"
Jul 17 01:50:55.822698 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.822639    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="30c738d7-6336-40f0-932a-67fe6d49a908" containerName="revoke-keys"
Jul 17 01:50:55.826728 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.826707    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"
Jul 17 01:50:55.832208 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.832175    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"]
Jul 17 01:50:55.891101 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.891016    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a90809-01ac-4217-ac05-8157326dbee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj\" (UID: \"22a90809-01ac-4217-ac05-8157326dbee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"
Jul 17 01:50:55.992303 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.992263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a90809-01ac-4217-ac05-8157326dbee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj\" (UID: \"22a90809-01ac-4217-ac05-8157326dbee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"
Jul 17 01:50:55.992960 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:55.992940    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a90809-01ac-4217-ac05-8157326dbee5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj\" (UID: \"22a90809-01ac-4217-ac05-8157326dbee5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"
Jul 17 01:50:56.138532 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:56.138475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"
Jul 17 01:50:56.272911 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:56.272887    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"]
Jul 17 01:50:56.275378 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:50:56.275349    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a90809_01ac_4217_ac05_8157326dbee5.slice/crio-4f63b119d511ce8f2d3d5442a2d5c1f7593994b2d8804ae41f78dc6b5ad544cc WatchSource:0}: Error finding container 4f63b119d511ce8f2d3d5442a2d5c1f7593994b2d8804ae41f78dc6b5ad544cc: Status 404 returned error can't find the container with id 4f63b119d511ce8f2d3d5442a2d5c1f7593994b2d8804ae41f78dc6b5ad544cc
Jul 17 01:50:56.787425 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:56.787337    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" event={"ID":"22a90809-01ac-4217-ac05-8157326dbee5","Type":"ContainerStarted","Data":"1935981b2a70d739ace9237373b5032eb3a0c15e3c5b9420ab5b90098b0f51a8"}
Jul 17 01:50:56.787425 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:56.787388    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" event={"ID":"22a90809-01ac-4217-ac05-8157326dbee5","Type":"ContainerStarted","Data":"4f63b119d511ce8f2d3d5442a2d5c1f7593994b2d8804ae41f78dc6b5ad544cc"}
Jul 17 01:50:56.801617 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:56.801567    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" podStartSLOduration=1.801551964 podStartE2EDuration="1.801551964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:50:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:50:56.801298721 +0000 UTC m=+2264.304984587" watchObservedRunningTime="2026-07-17 01:50:56.801551964 +0000 UTC m=+2264.305237826"
Jul 17 01:50:56.807061 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:56.807038    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"]
Jul 17 01:50:56.941845 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:56.941820    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"
Jul 17 01:50:57.000986 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.000950    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/379f8806-8c0b-4c8e-bb62-03a89764ee77-maas-api-service-ca\") pod \"379f8806-8c0b-4c8e-bb62-03a89764ee77\" (UID: \"379f8806-8c0b-4c8e-bb62-03a89764ee77\") "
Jul 17 01:50:57.001349 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.001256    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/379f8806-8c0b-4c8e-bb62-03a89764ee77-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "379f8806-8c0b-4c8e-bb62-03a89764ee77" (UID: "379f8806-8c0b-4c8e-bb62-03a89764ee77"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:50:57.102042 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.101972    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/379f8806-8c0b-4c8e-bb62-03a89764ee77-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:50:57.794574 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.794542    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"
Jul 17 01:50:57.794755 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.794536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh" event={"ID":"379f8806-8c0b-4c8e-bb62-03a89764ee77","Type":"ContainerDied","Data":"67abc7aebfa19a0ed9cf065e24a43a06e5ab702f2c9bdc974b415b1c9c5c99dc"}
Jul 17 01:50:57.794755 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.794727    2582 scope.go:117] "RemoveContainer" containerID="8ffca540afadbca8851ddb468d494da67b02afeccee8ae861ab1b372558c1722"
Jul 17 01:50:57.796141 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.795997    2582 generic.go:356] "Generic (PLEG): container finished" podID="22a90809-01ac-4217-ac05-8157326dbee5" containerID="1935981b2a70d739ace9237373b5032eb3a0c15e3c5b9420ab5b90098b0f51a8" exitCode=6
Jul 17 01:50:57.796141 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.796053    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" event={"ID":"22a90809-01ac-4217-ac05-8157326dbee5","Type":"ContainerDied","Data":"1935981b2a70d739ace9237373b5032eb3a0c15e3c5b9420ab5b90098b0f51a8"}
Jul 17 01:50:57.796249 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.796236    2582 scope.go:117] "RemoveContainer" containerID="1935981b2a70d739ace9237373b5032eb3a0c15e3c5b9420ab5b90098b0f51a8"
Jul 17 01:50:57.825346 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.825319    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"]
Jul 17 01:50:57.828887 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:57.828864    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-hlhjh"]
Jul 17 01:50:58.803749 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:58.803659    2582 generic.go:356] "Generic (PLEG): container finished" podID="22a90809-01ac-4217-ac05-8157326dbee5" containerID="9b19b9a61f6fcf8e51086264f1988b6af25a0a318875ef8c1a2227d046bb3bdd" exitCode=6
Jul 17 01:50:58.803749 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:58.803705    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" event={"ID":"22a90809-01ac-4217-ac05-8157326dbee5","Type":"ContainerDied","Data":"9b19b9a61f6fcf8e51086264f1988b6af25a0a318875ef8c1a2227d046bb3bdd"}
Jul 17 01:50:58.804290 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:58.803773    2582 scope.go:117] "RemoveContainer" containerID="1935981b2a70d739ace9237373b5032eb3a0c15e3c5b9420ab5b90098b0f51a8"
Jul 17 01:50:58.804290 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:58.803962    2582 scope.go:117] "RemoveContainer" containerID="9b19b9a61f6fcf8e51086264f1988b6af25a0a318875ef8c1a2227d046bb3bdd"
Jul 17 01:50:58.804290 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:58.804247    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj_odh-ai-gateway-infra(22a90809-01ac-4217-ac05-8157326dbee5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" podUID="22a90809-01ac-4217-ac05-8157326dbee5"
Jul 17 01:50:58.990528 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:58.990473    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" path="/var/lib/kubelet/pods/379f8806-8c0b-4c8e-bb62-03a89764ee77/volumes"
Jul 17 01:50:59.811038 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:50:59.811014    2582 scope.go:117] "RemoveContainer" containerID="9b19b9a61f6fcf8e51086264f1988b6af25a0a318875ef8c1a2227d046bb3bdd"
Jul 17 01:50:59.811541 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:50:59.811256    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj_odh-ai-gateway-infra(22a90809-01ac-4217-ac05-8157326dbee5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" podUID="22a90809-01ac-4217-ac05-8157326dbee5"
Jul 17 01:51:12.986735 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:12.986709    2582 scope.go:117] "RemoveContainer" containerID="9b19b9a61f6fcf8e51086264f1988b6af25a0a318875ef8c1a2227d046bb3bdd"
Jul 17 01:51:13.878963 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:13.878930    2582 generic.go:356] "Generic (PLEG): container finished" podID="22a90809-01ac-4217-ac05-8157326dbee5" containerID="a6b22fd54c392acd04540f73a5ebd9d9c437c788eac1967db1dff5caa08437c4" exitCode=6
Jul 17 01:51:13.879125 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:13.879006    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" event={"ID":"22a90809-01ac-4217-ac05-8157326dbee5","Type":"ContainerDied","Data":"a6b22fd54c392acd04540f73a5ebd9d9c437c788eac1967db1dff5caa08437c4"}
Jul 17 01:51:13.879125 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:13.879072    2582 scope.go:117] "RemoveContainer" containerID="9b19b9a61f6fcf8e51086264f1988b6af25a0a318875ef8c1a2227d046bb3bdd"
Jul 17 01:51:13.879394 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:13.879379    2582 scope.go:117] "RemoveContainer" containerID="a6b22fd54c392acd04540f73a5ebd9d9c437c788eac1967db1dff5caa08437c4"
Jul 17 01:51:13.879618 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:51:13.879596    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj_odh-ai-gateway-infra(22a90809-01ac-4217-ac05-8157326dbee5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" podUID="22a90809-01ac-4217-ac05-8157326dbee5"
Jul 17 01:51:14.905748 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:14.905714    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"]
Jul 17 01:51:15.046141 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.046117    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"
Jul 17 01:51:15.163408 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.163332    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a90809-01ac-4217-ac05-8157326dbee5-maas-api-service-ca\") pod \"22a90809-01ac-4217-ac05-8157326dbee5\" (UID: \"22a90809-01ac-4217-ac05-8157326dbee5\") "
Jul 17 01:51:15.163713 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.163693    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22a90809-01ac-4217-ac05-8157326dbee5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "22a90809-01ac-4217-ac05-8157326dbee5" (UID: "22a90809-01ac-4217-ac05-8157326dbee5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:51:15.264912 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.264879    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22a90809-01ac-4217-ac05-8157326dbee5-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:51:15.891355 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.891328    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"
Jul 17 01:51:15.891355 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.891349    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj" event={"ID":"22a90809-01ac-4217-ac05-8157326dbee5","Type":"ContainerDied","Data":"4f63b119d511ce8f2d3d5442a2d5c1f7593994b2d8804ae41f78dc6b5ad544cc"}
Jul 17 01:51:15.891615 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.891405    2582 scope.go:117] "RemoveContainer" containerID="a6b22fd54c392acd04540f73a5ebd9d9c437c788eac1967db1dff5caa08437c4"
Jul 17 01:51:15.911661 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.911634    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"]
Jul 17 01:51:15.916384 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:15.916356    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-dpqvj"]
Jul 17 01:51:16.990753 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:16.990708    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22a90809-01ac-4217-ac05-8157326dbee5" path="/var/lib/kubelet/pods/22a90809-01ac-4217-ac05-8157326dbee5/volumes"
Jul 17 01:51:27.882209 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882173    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"]
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882544    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882555    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882573    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882578    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882586    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882592    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882603    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882608    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882614    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882620    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882627    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882632    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882683    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882691    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882698    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:27.882715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.882706    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:27.885743 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.885727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"
Jul 17 01:51:27.888186 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.888159    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:51:27.888186 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.888170    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:51:27.891884 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.891862    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"]
Jul 17 01:51:27.979975 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:27.979939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1a237a-e3c7-488e-ad85-f2d1918c02a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8\" (UID: \"9e1a237a-e3c7-488e-ad85-f2d1918c02a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"
Jul 17 01:51:28.080656 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:28.080622    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1a237a-e3c7-488e-ad85-f2d1918c02a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8\" (UID: \"9e1a237a-e3c7-488e-ad85-f2d1918c02a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"
Jul 17 01:51:28.081248 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:28.081227    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1a237a-e3c7-488e-ad85-f2d1918c02a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8\" (UID: \"9e1a237a-e3c7-488e-ad85-f2d1918c02a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"
Jul 17 01:51:28.197449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:28.197417    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"
Jul 17 01:51:28.327902 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:28.327838    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"]
Jul 17 01:51:28.330369 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:51:28.330339    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e1a237a_e3c7_488e_ad85_f2d1918c02a2.slice/crio-2d3535f10c0a3640766c12342b6c528dbdbd31f110d04875b76779af9e42326c WatchSource:0}: Error finding container 2d3535f10c0a3640766c12342b6c528dbdbd31f110d04875b76779af9e42326c: Status 404 returned error can't find the container with id 2d3535f10c0a3640766c12342b6c528dbdbd31f110d04875b76779af9e42326c
Jul 17 01:51:28.952433 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:28.952403    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" event={"ID":"9e1a237a-e3c7-488e-ad85-f2d1918c02a2","Type":"ContainerStarted","Data":"02e521e68f07d5069bcd19ccfbd8ea607b0c715ec7b9a4f01093de4fba6013bf"}
Jul 17 01:51:28.952876 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:28.952455    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" event={"ID":"9e1a237a-e3c7-488e-ad85-f2d1918c02a2","Type":"ContainerStarted","Data":"2d3535f10c0a3640766c12342b6c528dbdbd31f110d04875b76779af9e42326c"}
Jul 17 01:51:28.966847 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:28.966802    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" podStartSLOduration=1.966786979 podStartE2EDuration="1.966786979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:51:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:51:28.966009037 +0000 UTC m=+2296.469694901" watchObservedRunningTime="2026-07-17 01:51:28.966786979 +0000 UTC m=+2296.470472841"
Jul 17 01:51:29.958633 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:29.958595    2582 generic.go:356] "Generic (PLEG): container finished" podID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerID="02e521e68f07d5069bcd19ccfbd8ea607b0c715ec7b9a4f01093de4fba6013bf" exitCode=6
Jul 17 01:51:29.959025 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:29.958666    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" event={"ID":"9e1a237a-e3c7-488e-ad85-f2d1918c02a2","Type":"ContainerDied","Data":"02e521e68f07d5069bcd19ccfbd8ea607b0c715ec7b9a4f01093de4fba6013bf"}
Jul 17 01:51:29.959025 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:29.958924    2582 scope.go:117] "RemoveContainer" containerID="02e521e68f07d5069bcd19ccfbd8ea607b0c715ec7b9a4f01093de4fba6013bf"
Jul 17 01:51:30.964842 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:30.964796    2582 generic.go:356] "Generic (PLEG): container finished" podID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerID="f86bce861da36c1be72a4b6c95ab31d4c02dfbebe21c6ebe7106fb55a94bbd4b" exitCode=6
Jul 17 01:51:30.965287 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:30.964885    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" event={"ID":"9e1a237a-e3c7-488e-ad85-f2d1918c02a2","Type":"ContainerDied","Data":"f86bce861da36c1be72a4b6c95ab31d4c02dfbebe21c6ebe7106fb55a94bbd4b"}
Jul 17 01:51:30.965287 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:30.964952    2582 scope.go:117] "RemoveContainer" containerID="02e521e68f07d5069bcd19ccfbd8ea607b0c715ec7b9a4f01093de4fba6013bf"
Jul 17 01:51:30.965287 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:30.965145    2582 scope.go:117] "RemoveContainer" containerID="f86bce861da36c1be72a4b6c95ab31d4c02dfbebe21c6ebe7106fb55a94bbd4b"
Jul 17 01:51:30.965449 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:51:30.965386    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8_odh-ai-gateway-infra(9e1a237a-e3c7-488e-ad85-f2d1918c02a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2"
Jul 17 01:51:31.971808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:31.971780    2582 scope.go:117] "RemoveContainer" containerID="f86bce861da36c1be72a4b6c95ab31d4c02dfbebe21c6ebe7106fb55a94bbd4b"
Jul 17 01:51:31.972268 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:51:31.972040    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8_odh-ai-gateway-infra(9e1a237a-e3c7-488e-ad85-f2d1918c02a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2"
Jul 17 01:51:39.778287 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:39.778248    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"]
Jul 17 01:51:39.778704 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:39.778671    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="379f8806-8c0b-4c8e-bb62-03a89764ee77" containerName="revoke-keys"
Jul 17 01:51:39.778704 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:39.778689    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="22a90809-01ac-4217-ac05-8157326dbee5" containerName="revoke-keys"
Jul 17 01:51:39.782917 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:39.782901    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"
Jul 17 01:51:39.788548 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:39.788519    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"]
Jul 17 01:51:39.894054 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:39.894014    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2c775c-9d61-4e84-bb33-35e9b877ad14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj\" (UID: \"4f2c775c-9d61-4e84-bb33-35e9b877ad14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"
Jul 17 01:51:39.995027 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:39.994987    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2c775c-9d61-4e84-bb33-35e9b877ad14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj\" (UID: \"4f2c775c-9d61-4e84-bb33-35e9b877ad14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"
Jul 17 01:51:39.995726 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:39.995697    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2c775c-9d61-4e84-bb33-35e9b877ad14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj\" (UID: \"4f2c775c-9d61-4e84-bb33-35e9b877ad14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"
Jul 17 01:51:40.095683 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:40.095591    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"
Jul 17 01:51:40.231096 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:40.231073    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"]
Jul 17 01:51:40.233452 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:51:40.233422    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f2c775c_9d61_4e84_bb33_35e9b877ad14.slice/crio-726e932cc240e631cf05bcbbb18cb9f4449b83d9794ca32f0f628488f9600630 WatchSource:0}: Error finding container 726e932cc240e631cf05bcbbb18cb9f4449b83d9794ca32f0f628488f9600630: Status 404 returned error can't find the container with id 726e932cc240e631cf05bcbbb18cb9f4449b83d9794ca32f0f628488f9600630
Jul 17 01:51:41.017528 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:41.017476    2582 generic.go:356] "Generic (PLEG): container finished" podID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerID="49252eae9e6aaf52ae88510254724f1fc71ad376aac4cc45bdbbeb6964928368" exitCode=6
Jul 17 01:51:41.017914 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:41.017524    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj" event={"ID":"4f2c775c-9d61-4e84-bb33-35e9b877ad14","Type":"ContainerDied","Data":"49252eae9e6aaf52ae88510254724f1fc71ad376aac4cc45bdbbeb6964928368"}
Jul 17 01:51:41.017914 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:41.017578    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj" event={"ID":"4f2c775c-9d61-4e84-bb33-35e9b877ad14","Type":"ContainerStarted","Data":"726e932cc240e631cf05bcbbb18cb9f4449b83d9794ca32f0f628488f9600630"}
Jul 17 01:51:41.017914 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:41.017733    2582 scope.go:117] "RemoveContainer" containerID="49252eae9e6aaf52ae88510254724f1fc71ad376aac4cc45bdbbeb6964928368"
Jul 17 01:51:42.025206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:42.025173    2582 generic.go:356] "Generic (PLEG): container finished" podID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerID="e19fc2bba660688361fb9800b5c0589d6ea51b07b80e0da4f5419034c86e5dcc" exitCode=6
Jul 17 01:51:42.025683 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:42.025262    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj" event={"ID":"4f2c775c-9d61-4e84-bb33-35e9b877ad14","Type":"ContainerDied","Data":"e19fc2bba660688361fb9800b5c0589d6ea51b07b80e0da4f5419034c86e5dcc"}
Jul 17 01:51:42.025683 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:42.025322    2582 scope.go:117] "RemoveContainer" containerID="49252eae9e6aaf52ae88510254724f1fc71ad376aac4cc45bdbbeb6964928368"
Jul 17 01:51:42.025683 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:42.025554    2582 scope.go:117] "RemoveContainer" containerID="e19fc2bba660688361fb9800b5c0589d6ea51b07b80e0da4f5419034c86e5dcc"
Jul 17 01:51:42.025854 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:51:42.025752    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj_odh-ai-gateway-infra(4f2c775c-9d61-4e84-bb33-35e9b877ad14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14"
Jul 17 01:51:43.030907 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:43.030877    2582 scope.go:117] "RemoveContainer" containerID="e19fc2bba660688361fb9800b5c0589d6ea51b07b80e0da4f5419034c86e5dcc"
Jul 17 01:51:43.031312 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:51:43.031054    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj_odh-ai-gateway-infra(4f2c775c-9d61-4e84-bb33-35e9b877ad14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14"
Jul 17 01:51:45.985259 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:45.985230    2582 scope.go:117] "RemoveContainer" containerID="f86bce861da36c1be72a4b6c95ab31d4c02dfbebe21c6ebe7106fb55a94bbd4b"
Jul 17 01:51:47.051563 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:47.051449    2582 generic.go:356] "Generic (PLEG): container finished" podID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerID="15b29f77e1c4d22fb328734f63ce795c9084a47eccd91138e1ac569d7186dcfc" exitCode=6
Jul 17 01:51:47.051563 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:47.051525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" event={"ID":"9e1a237a-e3c7-488e-ad85-f2d1918c02a2","Type":"ContainerDied","Data":"15b29f77e1c4d22fb328734f63ce795c9084a47eccd91138e1ac569d7186dcfc"}
Jul 17 01:51:47.052010 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:47.051588    2582 scope.go:117] "RemoveContainer" containerID="f86bce861da36c1be72a4b6c95ab31d4c02dfbebe21c6ebe7106fb55a94bbd4b"
Jul 17 01:51:47.052010 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:47.051914    2582 scope.go:117] "RemoveContainer" containerID="15b29f77e1c4d22fb328734f63ce795c9084a47eccd91138e1ac569d7186dcfc"
Jul 17 01:51:47.052168 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:51:47.052147    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8_odh-ai-gateway-infra(9e1a237a-e3c7-488e-ad85-f2d1918c02a2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2"
Jul 17 01:51:48.074760 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:48.074728    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"]
Jul 17 01:51:48.211941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:48.211907    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"
Jul 17 01:51:48.372585 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:48.372476    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1a237a-e3c7-488e-ad85-f2d1918c02a2-maas-api-service-ca\") pod \"9e1a237a-e3c7-488e-ad85-f2d1918c02a2\" (UID: \"9e1a237a-e3c7-488e-ad85-f2d1918c02a2\") "
Jul 17 01:51:48.372882 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:48.372856    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e1a237a-e3c7-488e-ad85-f2d1918c02a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e1a237a-e3c7-488e-ad85-f2d1918c02a2" (UID: "9e1a237a-e3c7-488e-ad85-f2d1918c02a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:51:48.473757 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:48.473716    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e1a237a-e3c7-488e-ad85-f2d1918c02a2-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:51:49.063458 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:49.063429    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"
Jul 17 01:51:49.063670 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:49.063469    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8" event={"ID":"9e1a237a-e3c7-488e-ad85-f2d1918c02a2","Type":"ContainerDied","Data":"2d3535f10c0a3640766c12342b6c528dbdbd31f110d04875b76779af9e42326c"}
Jul 17 01:51:49.063670 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:49.063547    2582 scope.go:117] "RemoveContainer" containerID="15b29f77e1c4d22fb328734f63ce795c9084a47eccd91138e1ac569d7186dcfc"
Jul 17 01:51:49.079348 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:49.079324    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"]
Jul 17 01:51:49.084226 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:49.084204    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-qk6c8"]
Jul 17 01:51:50.989941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:50.989907    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" path="/var/lib/kubelet/pods/9e1a237a-e3c7-488e-ad85-f2d1918c02a2/volumes"
Jul 17 01:51:55.984712 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:55.984683    2582 scope.go:117] "RemoveContainer" containerID="e19fc2bba660688361fb9800b5c0589d6ea51b07b80e0da4f5419034c86e5dcc"
Jul 17 01:51:57.101110 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:57.101079    2582 generic.go:356] "Generic (PLEG): container finished" podID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerID="6ff4eca0da747cb63c3e3895d587ca1477b80de5a2f9057040f8f4ab617e8786" exitCode=6
Jul 17 01:51:57.101536 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:57.101152    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj" event={"ID":"4f2c775c-9d61-4e84-bb33-35e9b877ad14","Type":"ContainerDied","Data":"6ff4eca0da747cb63c3e3895d587ca1477b80de5a2f9057040f8f4ab617e8786"}
Jul 17 01:51:57.101536 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:57.101216    2582 scope.go:117] "RemoveContainer" containerID="e19fc2bba660688361fb9800b5c0589d6ea51b07b80e0da4f5419034c86e5dcc"
Jul 17 01:51:57.101647 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:57.101601    2582 scope.go:117] "RemoveContainer" containerID="6ff4eca0da747cb63c3e3895d587ca1477b80de5a2f9057040f8f4ab617e8786"
Jul 17 01:51:57.101826 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:51:57.101806    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj_odh-ai-gateway-infra(4f2c775c-9d61-4e84-bb33-35e9b877ad14)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14"
Jul 17 01:51:58.125105 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:58.125066    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"]
Jul 17 01:51:58.265081 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:58.265051    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"
Jul 17 01:51:58.365911 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:58.365879    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2c775c-9d61-4e84-bb33-35e9b877ad14-maas-api-service-ca\") pod \"4f2c775c-9d61-4e84-bb33-35e9b877ad14\" (UID: \"4f2c775c-9d61-4e84-bb33-35e9b877ad14\") "
Jul 17 01:51:58.366261 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:58.366235    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f2c775c-9d61-4e84-bb33-35e9b877ad14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4f2c775c-9d61-4e84-bb33-35e9b877ad14" (UID: "4f2c775c-9d61-4e84-bb33-35e9b877ad14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:51:58.467044 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:58.467014    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f2c775c-9d61-4e84-bb33-35e9b877ad14-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:51:59.112639 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:59.112603    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj" event={"ID":"4f2c775c-9d61-4e84-bb33-35e9b877ad14","Type":"ContainerDied","Data":"726e932cc240e631cf05bcbbb18cb9f4449b83d9794ca32f0f628488f9600630"}
Jul 17 01:51:59.112639 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:59.112639    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"
Jul 17 01:51:59.112984 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:59.112676    2582 scope.go:117] "RemoveContainer" containerID="6ff4eca0da747cb63c3e3895d587ca1477b80de5a2f9057040f8f4ab617e8786"
Jul 17 01:51:59.128238 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:59.128210    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"]
Jul 17 01:51:59.135839 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:51:59.135814    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-p78nj"]
Jul 17 01:52:00.990036 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:00.990001    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" path="/var/lib/kubelet/pods/4f2c775c-9d61-4e84-bb33-35e9b877ad14/volumes"
Jul 17 01:52:27.926325 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926290    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"]
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926676    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926687    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926694    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926699    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926721    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926726    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926734    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926739    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926785    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926792    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926800    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:27.926808 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.926806    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:27.929604 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.929588    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"
Jul 17 01:52:27.932018 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.931998    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:52:27.932156 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.932004    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:52:27.936637 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:27.936553    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"]
Jul 17 01:52:28.026898 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:28.026848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ff4848-4dfc-4ae6-98d7-5269baf8268e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn\" (UID: \"b9ff4848-4dfc-4ae6-98d7-5269baf8268e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"
Jul 17 01:52:28.127345 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:28.127303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ff4848-4dfc-4ae6-98d7-5269baf8268e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn\" (UID: \"b9ff4848-4dfc-4ae6-98d7-5269baf8268e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"
Jul 17 01:52:28.127960 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:28.127939    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ff4848-4dfc-4ae6-98d7-5269baf8268e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn\" (UID: \"b9ff4848-4dfc-4ae6-98d7-5269baf8268e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"
Jul 17 01:52:28.241480 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:28.241390    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"
Jul 17 01:52:28.588593 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:28.588484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"]
Jul 17 01:52:28.591466 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:52:28.591441    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ff4848_4dfc_4ae6_98d7_5269baf8268e.slice/crio-2b4b25dae7b05fc5a5b8fa1df04e97389982ceee734a8be9978ef1d406f29782 WatchSource:0}: Error finding container 2b4b25dae7b05fc5a5b8fa1df04e97389982ceee734a8be9978ef1d406f29782: Status 404 returned error can't find the container with id 2b4b25dae7b05fc5a5b8fa1df04e97389982ceee734a8be9978ef1d406f29782
Jul 17 01:52:29.261750 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.261713    2582 generic.go:356] "Generic (PLEG): container finished" podID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerID="895afed73f90e4b4402ae62b77574422191888d01e5d8378e33ed59ed31838f9" exitCode=6
Jul 17 01:52:29.262107 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.261760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" event={"ID":"b9ff4848-4dfc-4ae6-98d7-5269baf8268e","Type":"ContainerDied","Data":"895afed73f90e4b4402ae62b77574422191888d01e5d8378e33ed59ed31838f9"}
Jul 17 01:52:29.262107 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.261825    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" event={"ID":"b9ff4848-4dfc-4ae6-98d7-5269baf8268e","Type":"ContainerStarted","Data":"2b4b25dae7b05fc5a5b8fa1df04e97389982ceee734a8be9978ef1d406f29782"}
Jul 17 01:52:29.262107 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.262036    2582 scope.go:117] "RemoveContainer" containerID="895afed73f90e4b4402ae62b77574422191888d01e5d8378e33ed59ed31838f9"
Jul 17 01:52:29.818006 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.817926    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"]
Jul 17 01:52:29.818300 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.818288    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:29.818345 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.818301    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:29.818345 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.818309    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:29.818345 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.818314    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:29.818438 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.818375    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f2c775c-9d61-4e84-bb33-35e9b877ad14" containerName="revoke-keys"
Jul 17 01:52:29.818438 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.818385    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e1a237a-e3c7-488e-ad85-f2d1918c02a2" containerName="revoke-keys"
Jul 17 01:52:29.821309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.821293    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"
Jul 17 01:52:29.826675 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.826605    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"]
Jul 17 01:52:29.944600 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:29.944559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9200685e-2c33-4ba4-9d8e-1e86b1c3d703-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8\" (UID: \"9200685e-2c33-4ba4-9d8e-1e86b1c3d703\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"
Jul 17 01:52:30.045667 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:30.045626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9200685e-2c33-4ba4-9d8e-1e86b1c3d703-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8\" (UID: \"9200685e-2c33-4ba4-9d8e-1e86b1c3d703\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"
Jul 17 01:52:30.046245 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:30.046225    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9200685e-2c33-4ba4-9d8e-1e86b1c3d703-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8\" (UID: \"9200685e-2c33-4ba4-9d8e-1e86b1c3d703\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"
Jul 17 01:52:30.135236 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:30.135146    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"
Jul 17 01:52:30.268011 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:30.267980    2582 generic.go:356] "Generic (PLEG): container finished" podID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerID="042b2341f7b8dcc15fb0a77eb2bbc22c18954ad3e6bb5eb43b06613ffd1e6a61" exitCode=6
Jul 17 01:52:30.268416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:30.268056    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" event={"ID":"b9ff4848-4dfc-4ae6-98d7-5269baf8268e","Type":"ContainerDied","Data":"042b2341f7b8dcc15fb0a77eb2bbc22c18954ad3e6bb5eb43b06613ffd1e6a61"}
Jul 17 01:52:30.268416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:30.268117    2582 scope.go:117] "RemoveContainer" containerID="895afed73f90e4b4402ae62b77574422191888d01e5d8378e33ed59ed31838f9"
Jul 17 01:52:30.268416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:30.268354    2582 scope.go:117] "RemoveContainer" containerID="042b2341f7b8dcc15fb0a77eb2bbc22c18954ad3e6bb5eb43b06613ffd1e6a61"
Jul 17 01:52:30.268638 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:52:30.268619    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn_odh-ai-gateway-infra(b9ff4848-4dfc-4ae6-98d7-5269baf8268e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e"
Jul 17 01:52:30.273600 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:30.273567    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"]
Jul 17 01:52:30.276193 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:52:30.276167    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9200685e_2c33_4ba4_9d8e_1e86b1c3d703.slice/crio-23f93273df562d8a96231eb8e9c7edc3dbeafc72a5aecc9248a3943edf96d120 WatchSource:0}: Error finding container 23f93273df562d8a96231eb8e9c7edc3dbeafc72a5aecc9248a3943edf96d120: Status 404 returned error can't find the container with id 23f93273df562d8a96231eb8e9c7edc3dbeafc72a5aecc9248a3943edf96d120
Jul 17 01:52:31.274565 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:31.274446    2582 scope.go:117] "RemoveContainer" containerID="042b2341f7b8dcc15fb0a77eb2bbc22c18954ad3e6bb5eb43b06613ffd1e6a61"
Jul 17 01:52:31.275003 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:52:31.274680    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn_odh-ai-gateway-infra(b9ff4848-4dfc-4ae6-98d7-5269baf8268e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e"
Jul 17 01:52:31.275688 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:31.275670    2582 generic.go:356] "Generic (PLEG): container finished" podID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerID="fb572a35a022ba42ada0271a054c6c528d0900acb3b90a9b6d99620aa3bf0710" exitCode=6
Jul 17 01:52:31.275765 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:31.275718    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" event={"ID":"9200685e-2c33-4ba4-9d8e-1e86b1c3d703","Type":"ContainerDied","Data":"fb572a35a022ba42ada0271a054c6c528d0900acb3b90a9b6d99620aa3bf0710"}
Jul 17 01:52:31.275765 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:31.275760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" event={"ID":"9200685e-2c33-4ba4-9d8e-1e86b1c3d703","Type":"ContainerStarted","Data":"23f93273df562d8a96231eb8e9c7edc3dbeafc72a5aecc9248a3943edf96d120"}
Jul 17 01:52:31.275970 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:31.275921    2582 scope.go:117] "RemoveContainer" containerID="fb572a35a022ba42ada0271a054c6c528d0900acb3b90a9b6d99620aa3bf0710"
Jul 17 01:52:32.282702 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:32.282669    2582 generic.go:356] "Generic (PLEG): container finished" podID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerID="3e97d47ec2be44eebb1405f1eb24c4bf004c6d40a677038d562ade8d698bc074" exitCode=6
Jul 17 01:52:32.283126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:32.282725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" event={"ID":"9200685e-2c33-4ba4-9d8e-1e86b1c3d703","Type":"ContainerDied","Data":"3e97d47ec2be44eebb1405f1eb24c4bf004c6d40a677038d562ade8d698bc074"}
Jul 17 01:52:32.283126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:32.282778    2582 scope.go:117] "RemoveContainer" containerID="fb572a35a022ba42ada0271a054c6c528d0900acb3b90a9b6d99620aa3bf0710"
Jul 17 01:52:32.283126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:32.282939    2582 scope.go:117] "RemoveContainer" containerID="3e97d47ec2be44eebb1405f1eb24c4bf004c6d40a677038d562ade8d698bc074"
Jul 17 01:52:32.283228 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:52:32.283191    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8_odh-ai-gateway-infra(9200685e-2c33-4ba4-9d8e-1e86b1c3d703)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703"
Jul 17 01:52:33.289151 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:33.289119    2582 scope.go:117] "RemoveContainer" containerID="3e97d47ec2be44eebb1405f1eb24c4bf004c6d40a677038d562ade8d698bc074"
Jul 17 01:52:33.289568 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:52:33.289324    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8_odh-ai-gateway-infra(9200685e-2c33-4ba4-9d8e-1e86b1c3d703)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703"
Jul 17 01:52:43.984822 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:43.984784    2582 scope.go:117] "RemoveContainer" containerID="042b2341f7b8dcc15fb0a77eb2bbc22c18954ad3e6bb5eb43b06613ffd1e6a61"
Jul 17 01:52:43.985255 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:43.984898    2582 scope.go:117] "RemoveContainer" containerID="3e97d47ec2be44eebb1405f1eb24c4bf004c6d40a677038d562ade8d698bc074"
Jul 17 01:52:44.342664 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:44.342626    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" event={"ID":"b9ff4848-4dfc-4ae6-98d7-5269baf8268e","Type":"ContainerStarted","Data":"6b347bf3bad021a406392b47f30aff2c060a8f5f9e967c493ca985f7471a230e"}
Jul 17 01:52:44.344415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:44.344389    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" event={"ID":"9200685e-2c33-4ba4-9d8e-1e86b1c3d703","Type":"ContainerStarted","Data":"6d60540d70cba006887429a75d48c1259a7709c01c4ad4d4ab7036a2ec62ebfa"}
Jul 17 01:52:44.359654 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:44.359606    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" podStartSLOduration=17.359589455 podStartE2EDuration="17.359589455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:52:44.358575471 +0000 UTC m=+2371.862261329" watchObservedRunningTime="2026-07-17 01:52:44.359589455 +0000 UTC m=+2371.863275317"
Jul 17 01:52:44.372441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:44.372266    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" podStartSLOduration=15.372249304 podStartE2EDuration="15.372249304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:52:44.372219921 +0000 UTC m=+2371.875905784" watchObservedRunningTime="2026-07-17 01:52:44.372249304 +0000 UTC m=+2371.875935145"
Jul 17 01:52:45.351272 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.351238    2582 generic.go:356] "Generic (PLEG): container finished" podID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerID="6b347bf3bad021a406392b47f30aff2c060a8f5f9e967c493ca985f7471a230e" exitCode=6
Jul 17 01:52:45.351773 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.351317    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" event={"ID":"b9ff4848-4dfc-4ae6-98d7-5269baf8268e","Type":"ContainerDied","Data":"6b347bf3bad021a406392b47f30aff2c060a8f5f9e967c493ca985f7471a230e"}
Jul 17 01:52:45.351773 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.351380    2582 scope.go:117] "RemoveContainer" containerID="042b2341f7b8dcc15fb0a77eb2bbc22c18954ad3e6bb5eb43b06613ffd1e6a61"
Jul 17 01:52:45.351773 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.351719    2582 scope.go:117] "RemoveContainer" containerID="6b347bf3bad021a406392b47f30aff2c060a8f5f9e967c493ca985f7471a230e"
Jul 17 01:52:45.351976 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:52:45.351941    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn_odh-ai-gateway-infra(b9ff4848-4dfc-4ae6-98d7-5269baf8268e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e"
Jul 17 01:52:45.353378 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.353356    2582 generic.go:356] "Generic (PLEG): container finished" podID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerID="6d60540d70cba006887429a75d48c1259a7709c01c4ad4d4ab7036a2ec62ebfa" exitCode=6
Jul 17 01:52:45.353457 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.353433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" event={"ID":"9200685e-2c33-4ba4-9d8e-1e86b1c3d703","Type":"ContainerDied","Data":"6d60540d70cba006887429a75d48c1259a7709c01c4ad4d4ab7036a2ec62ebfa"}
Jul 17 01:52:45.353715 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.353702    2582 scope.go:117] "RemoveContainer" containerID="6d60540d70cba006887429a75d48c1259a7709c01c4ad4d4ab7036a2ec62ebfa"
Jul 17 01:52:45.353873 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:52:45.353857    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8_odh-ai-gateway-infra(9200685e-2c33-4ba4-9d8e-1e86b1c3d703)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703"
Jul 17 01:52:45.364652 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.364631    2582 scope.go:117] "RemoveContainer" containerID="3e97d47ec2be44eebb1405f1eb24c4bf004c6d40a677038d562ade8d698bc074"
Jul 17 01:52:45.376096 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.376070    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"]
Jul 17 01:52:45.382356 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:45.382334    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"]
Jul 17 01:52:46.523217 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:46.523192    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"
Jul 17 01:52:46.525309 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:46.525288    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"
Jul 17 01:52:46.695510 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:46.695462    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ff4848-4dfc-4ae6-98d7-5269baf8268e-maas-api-service-ca\") pod \"b9ff4848-4dfc-4ae6-98d7-5269baf8268e\" (UID: \"b9ff4848-4dfc-4ae6-98d7-5269baf8268e\") "
Jul 17 01:52:46.695510 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:46.695510    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9200685e-2c33-4ba4-9d8e-1e86b1c3d703-maas-api-service-ca\") pod \"9200685e-2c33-4ba4-9d8e-1e86b1c3d703\" (UID: \"9200685e-2c33-4ba4-9d8e-1e86b1c3d703\") "
Jul 17 01:52:46.695878 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:46.695852    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9200685e-2c33-4ba4-9d8e-1e86b1c3d703-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9200685e-2c33-4ba4-9d8e-1e86b1c3d703" (UID: "9200685e-2c33-4ba4-9d8e-1e86b1c3d703"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:52:46.695953 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:46.695860    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9ff4848-4dfc-4ae6-98d7-5269baf8268e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b9ff4848-4dfc-4ae6-98d7-5269baf8268e" (UID: "b9ff4848-4dfc-4ae6-98d7-5269baf8268e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:52:46.797003 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:46.796959    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9ff4848-4dfc-4ae6-98d7-5269baf8268e-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:52:46.797003 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:46.796995    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9200685e-2c33-4ba4-9d8e-1e86b1c3d703-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:52:47.367008 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.366911    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"
Jul 17 01:52:47.367008 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.366928    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn" event={"ID":"b9ff4848-4dfc-4ae6-98d7-5269baf8268e","Type":"ContainerDied","Data":"2b4b25dae7b05fc5a5b8fa1df04e97389982ceee734a8be9978ef1d406f29782"}
Jul 17 01:52:47.367008 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.366998    2582 scope.go:117] "RemoveContainer" containerID="6b347bf3bad021a406392b47f30aff2c060a8f5f9e967c493ca985f7471a230e"
Jul 17 01:52:47.368451 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.368430    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8" event={"ID":"9200685e-2c33-4ba4-9d8e-1e86b1c3d703","Type":"ContainerDied","Data":"23f93273df562d8a96231eb8e9c7edc3dbeafc72a5aecc9248a3943edf96d120"}
Jul 17 01:52:47.368550 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.368525    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"
Jul 17 01:52:47.376874 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.376846    2582 scope.go:117] "RemoveContainer" containerID="6d60540d70cba006887429a75d48c1259a7709c01c4ad4d4ab7036a2ec62ebfa"
Jul 17 01:52:47.392135 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.392106    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"]
Jul 17 01:52:47.398884 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.398859    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-2hrsn"]
Jul 17 01:52:47.410805 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.410780    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"]
Jul 17 01:52:47.412799 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:47.412778    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-bqnx8"]
Jul 17 01:52:48.990417 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:48.990381    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" path="/var/lib/kubelet/pods/9200685e-2c33-4ba4-9d8e-1e86b1c3d703/volumes"
Jul 17 01:52:48.990822 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:48.990760    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" path="/var/lib/kubelet/pods/b9ff4848-4dfc-4ae6-98d7-5269baf8268e/volumes"
Jul 17 01:52:53.320980 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.320948    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"]
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321299    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321309    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321323    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321328    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321336    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321342    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321353    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321358    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321367    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:52:53.321672 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321372    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:52:53.321672 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321429    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321672 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321437    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321672 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321444    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9ff4848-4dfc-4ae6-98d7-5269baf8268e" containerName="revoke-keys"
Jul 17 01:52:53.321672 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321452    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:52:53.321672 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321457    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:52:53.321672 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.321463    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:52:53.324273 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.324258    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"
Jul 17 01:52:53.326649 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.326623    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:52:53.326778 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.326625    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:52:53.329932 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.329909    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"]
Jul 17 01:52:53.355291 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.355253    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/597450fa-5820-4e80-a9ec-ecd14836d36a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc\" (UID: \"597450fa-5820-4e80-a9ec-ecd14836d36a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"
Jul 17 01:52:53.456572 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.456539    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/597450fa-5820-4e80-a9ec-ecd14836d36a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc\" (UID: \"597450fa-5820-4e80-a9ec-ecd14836d36a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"
Jul 17 01:52:53.457161 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.457143    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/597450fa-5820-4e80-a9ec-ecd14836d36a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc\" (UID: \"597450fa-5820-4e80-a9ec-ecd14836d36a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"
Jul 17 01:52:53.636531 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.636432    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"
Jul 17 01:52:53.765391 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:53.765325    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"]
Jul 17 01:52:53.768127 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:52:53.768097    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597450fa_5820_4e80_a9ec_ecd14836d36a.slice/crio-5aedbf7ca017c6c7053bd9b47f5dacf74d61f3ed2d783ebdbd64ee2d3f2a4b20 WatchSource:0}: Error finding container 5aedbf7ca017c6c7053bd9b47f5dacf74d61f3ed2d783ebdbd64ee2d3f2a4b20: Status 404 returned error can't find the container with id 5aedbf7ca017c6c7053bd9b47f5dacf74d61f3ed2d783ebdbd64ee2d3f2a4b20
Jul 17 01:52:54.404015 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:54.403981    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" event={"ID":"597450fa-5820-4e80-a9ec-ecd14836d36a","Type":"ContainerStarted","Data":"9bb03e2412e2d688344eee976c82bd43f1e359b15fdb609e6b43bb4fe62b22ef"}
Jul 17 01:52:54.404583 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:54.404035    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" event={"ID":"597450fa-5820-4e80-a9ec-ecd14836d36a","Type":"ContainerStarted","Data":"5aedbf7ca017c6c7053bd9b47f5dacf74d61f3ed2d783ebdbd64ee2d3f2a4b20"}
Jul 17 01:52:54.419080 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:54.419033    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" podStartSLOduration=1.419014811 podStartE2EDuration="1.419014811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:52:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:52:54.417938056 +0000 UTC m=+2381.921623919" watchObservedRunningTime="2026-07-17 01:52:54.419014811 +0000 UTC m=+2381.922700673"
Jul 17 01:52:55.410868 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:55.410832    2582 generic.go:356] "Generic (PLEG): container finished" podID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerID="9bb03e2412e2d688344eee976c82bd43f1e359b15fdb609e6b43bb4fe62b22ef" exitCode=7
Jul 17 01:52:55.411278 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:55.410872    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" event={"ID":"597450fa-5820-4e80-a9ec-ecd14836d36a","Type":"ContainerDied","Data":"9bb03e2412e2d688344eee976c82bd43f1e359b15fdb609e6b43bb4fe62b22ef"}
Jul 17 01:52:55.411278 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:55.411084    2582 scope.go:117] "RemoveContainer" containerID="9bb03e2412e2d688344eee976c82bd43f1e359b15fdb609e6b43bb4fe62b22ef"
Jul 17 01:52:56.416989 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:56.416901    2582 generic.go:356] "Generic (PLEG): container finished" podID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerID="26b9643948ddef3694e8eff630e264f9fcdab331ff5cc4ef2f1037358ea94f3c" exitCode=7
Jul 17 01:52:56.417450 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:56.416984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" event={"ID":"597450fa-5820-4e80-a9ec-ecd14836d36a","Type":"ContainerDied","Data":"26b9643948ddef3694e8eff630e264f9fcdab331ff5cc4ef2f1037358ea94f3c"}
Jul 17 01:52:56.417450 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:56.417038    2582 scope.go:117] "RemoveContainer" containerID="9bb03e2412e2d688344eee976c82bd43f1e359b15fdb609e6b43bb4fe62b22ef"
Jul 17 01:52:56.417450 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:56.417332    2582 scope.go:117] "RemoveContainer" containerID="26b9643948ddef3694e8eff630e264f9fcdab331ff5cc4ef2f1037358ea94f3c"
Jul 17 01:52:56.417657 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:52:56.417604    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc_odh-ai-gateway-infra(597450fa-5820-4e80-a9ec-ecd14836d36a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a"
Jul 17 01:52:57.424020 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:52:57.423990    2582 scope.go:117] "RemoveContainer" containerID="26b9643948ddef3694e8eff630e264f9fcdab331ff5cc4ef2f1037358ea94f3c"
Jul 17 01:52:57.424434 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:52:57.424164    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc_odh-ai-gateway-infra(597450fa-5820-4e80-a9ec-ecd14836d36a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a"
Jul 17 01:53:08.985558 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:08.985522    2582 scope.go:117] "RemoveContainer" containerID="26b9643948ddef3694e8eff630e264f9fcdab331ff5cc4ef2f1037358ea94f3c"
Jul 17 01:53:09.481269 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:09.481237    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" event={"ID":"597450fa-5820-4e80-a9ec-ecd14836d36a","Type":"ContainerStarted","Data":"7c117e28722cd0369346fb4992569c8cfb8de9b4b5f100db8357ea25f9d952b3"}
Jul 17 01:53:10.487416 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:10.487377    2582 generic.go:356] "Generic (PLEG): container finished" podID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerID="7c117e28722cd0369346fb4992569c8cfb8de9b4b5f100db8357ea25f9d952b3" exitCode=7
Jul 17 01:53:10.487938 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:10.487449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" event={"ID":"597450fa-5820-4e80-a9ec-ecd14836d36a","Type":"ContainerDied","Data":"7c117e28722cd0369346fb4992569c8cfb8de9b4b5f100db8357ea25f9d952b3"}
Jul 17 01:53:10.487938 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:10.487534    2582 scope.go:117] "RemoveContainer" containerID="26b9643948ddef3694e8eff630e264f9fcdab331ff5cc4ef2f1037358ea94f3c"
Jul 17 01:53:10.487938 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:10.487871    2582 scope.go:117] "RemoveContainer" containerID="7c117e28722cd0369346fb4992569c8cfb8de9b4b5f100db8357ea25f9d952b3"
Jul 17 01:53:10.488168 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:53:10.488146    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc_odh-ai-gateway-infra(597450fa-5820-4e80-a9ec-ecd14836d36a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a"
Jul 17 01:53:10.511570 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:10.511541    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"]
Jul 17 01:53:11.650794 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:11.650767    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"
Jul 17 01:53:11.725606 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:11.725566    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/597450fa-5820-4e80-a9ec-ecd14836d36a-maas-api-service-ca\") pod \"597450fa-5820-4e80-a9ec-ecd14836d36a\" (UID: \"597450fa-5820-4e80-a9ec-ecd14836d36a\") "
Jul 17 01:53:11.725914 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:11.725891    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/597450fa-5820-4e80-a9ec-ecd14836d36a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "597450fa-5820-4e80-a9ec-ecd14836d36a" (UID: "597450fa-5820-4e80-a9ec-ecd14836d36a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:53:11.826636 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:11.826553    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/597450fa-5820-4e80-a9ec-ecd14836d36a-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:53:12.500012 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:12.499975    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc" event={"ID":"597450fa-5820-4e80-a9ec-ecd14836d36a","Type":"ContainerDied","Data":"5aedbf7ca017c6c7053bd9b47f5dacf74d61f3ed2d783ebdbd64ee2d3f2a4b20"}
Jul 17 01:53:12.500206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:12.500050    2582 scope.go:117] "RemoveContainer" containerID="7c117e28722cd0369346fb4992569c8cfb8de9b4b5f100db8357ea25f9d952b3"
Jul 17 01:53:12.500206 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:12.499986    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"
Jul 17 01:53:12.519839 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:12.519812    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"]
Jul 17 01:53:12.527044 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:12.527016    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-lw4kc"]
Jul 17 01:53:12.991567 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:12.991534    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" path="/var/lib/kubelet/pods/597450fa-5820-4e80-a9ec-ecd14836d36a/volumes"
Jul 17 01:53:14.363476 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:14.363362    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:53:14.384171 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:14.368407    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:53:14.384171 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:14.370166    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:53:14.384171 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:14.375078    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:53:15.968193 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:15.968150    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:53:17.968601 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.968563    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"]
Jul 17 01:53:17.969030 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.968936    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:53:17.969030 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.968948    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:53:17.969030 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.968963    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:53:17.969030 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.968969    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:53:17.969030 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.968986    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:53:17.969030 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.968991    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9200685e-2c33-4ba4-9d8e-1e86b1c3d703" containerName="revoke-keys"
Jul 17 01:53:17.969251 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.969045    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:53:17.969251 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.969056    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:53:17.969251 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.969063    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:53:17.971926 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.971912    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"
Jul 17 01:53:17.974199 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.974180    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:53:17.974300 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.974180    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:53:17.979931 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:17.979907    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"]
Jul 17 01:53:18.084842 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:18.084786    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/133d1527-3ae0-4fbc-9195-8e3809969c91-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8\" (UID: \"133d1527-3ae0-4fbc-9195-8e3809969c91\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"
Jul 17 01:53:18.185902 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:18.185864    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/133d1527-3ae0-4fbc-9195-8e3809969c91-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8\" (UID: \"133d1527-3ae0-4fbc-9195-8e3809969c91\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"
Jul 17 01:53:18.186476 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:18.186455    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/133d1527-3ae0-4fbc-9195-8e3809969c91-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8\" (UID: \"133d1527-3ae0-4fbc-9195-8e3809969c91\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"
Jul 17 01:53:18.283901 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:18.283816    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"
Jul 17 01:53:18.415258 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:18.415236    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"]
Jul 17 01:53:18.417391 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:53:18.417361    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133d1527_3ae0_4fbc_9195_8e3809969c91.slice/crio-153694e7976b735573c7c167363c3dcb458a900b7a94a3aacb6c678af26f3f5e WatchSource:0}: Error finding container 153694e7976b735573c7c167363c3dcb458a900b7a94a3aacb6c678af26f3f5e: Status 404 returned error can't find the container with id 153694e7976b735573c7c167363c3dcb458a900b7a94a3aacb6c678af26f3f5e
Jul 17 01:53:18.531395 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:18.531361    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" event={"ID":"133d1527-3ae0-4fbc-9195-8e3809969c91","Type":"ContainerStarted","Data":"153694e7976b735573c7c167363c3dcb458a900b7a94a3aacb6c678af26f3f5e"}
Jul 17 01:53:19.537743 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:19.537706    2582 generic.go:356] "Generic (PLEG): container finished" podID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerID="75fde6c18d893206f8dbbeb16b269e675ccb81379adaf3df5736187ac1d67a36" exitCode=6
Jul 17 01:53:19.538139 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:19.537780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" event={"ID":"133d1527-3ae0-4fbc-9195-8e3809969c91","Type":"ContainerDied","Data":"75fde6c18d893206f8dbbeb16b269e675ccb81379adaf3df5736187ac1d67a36"}
Jul 17 01:53:19.538139 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:19.538023    2582 scope.go:117] "RemoveContainer" containerID="75fde6c18d893206f8dbbeb16b269e675ccb81379adaf3df5736187ac1d67a36"
Jul 17 01:53:20.544357 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:20.544266    2582 generic.go:356] "Generic (PLEG): container finished" podID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerID="91f9edae9a11174797804b83c0487df755c68d82bb98ac8eca2025a2922a4f88" exitCode=6
Jul 17 01:53:20.544357 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:20.544348    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" event={"ID":"133d1527-3ae0-4fbc-9195-8e3809969c91","Type":"ContainerDied","Data":"91f9edae9a11174797804b83c0487df755c68d82bb98ac8eca2025a2922a4f88"}
Jul 17 01:53:20.544861 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:20.544412    2582 scope.go:117] "RemoveContainer" containerID="75fde6c18d893206f8dbbeb16b269e675ccb81379adaf3df5736187ac1d67a36"
Jul 17 01:53:20.544861 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:20.544638    2582 scope.go:117] "RemoveContainer" containerID="91f9edae9a11174797804b83c0487df755c68d82bb98ac8eca2025a2922a4f88"
Jul 17 01:53:20.544971 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:53:20.544872    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8_odh-ai-gateway-infra(133d1527-3ae0-4fbc-9195-8e3809969c91)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91"
Jul 17 01:53:21.550814 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:21.550789    2582 scope.go:117] "RemoveContainer" containerID="91f9edae9a11174797804b83c0487df755c68d82bb98ac8eca2025a2922a4f88"
Jul 17 01:53:21.551232 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:53:21.551000    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8_odh-ai-gateway-infra(133d1527-3ae0-4fbc-9195-8e3809969c91)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91"
Jul 17 01:53:34.984670 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:34.984575    2582 scope.go:117] "RemoveContainer" containerID="91f9edae9a11174797804b83c0487df755c68d82bb98ac8eca2025a2922a4f88"
Jul 17 01:53:35.624661 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:35.624570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" event={"ID":"133d1527-3ae0-4fbc-9195-8e3809969c91","Type":"ContainerStarted","Data":"1247da1b5aa1bce0ced4dadc584183204521b60c6bdbf72cb7434b06d0c5913c"}
Jul 17 01:53:35.639651 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:35.639602    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" podStartSLOduration=18.639586127 podStartE2EDuration="18.639586127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:53:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:53:35.638125325 +0000 UTC m=+2423.141811196" watchObservedRunningTime="2026-07-17 01:53:35.639586127 +0000 UTC m=+2423.143271990"
Jul 17 01:53:36.631459 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:36.631428    2582 generic.go:356] "Generic (PLEG): container finished" podID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerID="1247da1b5aa1bce0ced4dadc584183204521b60c6bdbf72cb7434b06d0c5913c" exitCode=6
Jul 17 01:53:36.631923 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:36.631470    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" event={"ID":"133d1527-3ae0-4fbc-9195-8e3809969c91","Type":"ContainerDied","Data":"1247da1b5aa1bce0ced4dadc584183204521b60c6bdbf72cb7434b06d0c5913c"}
Jul 17 01:53:36.631923 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:36.631546    2582 scope.go:117] "RemoveContainer" containerID="91f9edae9a11174797804b83c0487df755c68d82bb98ac8eca2025a2922a4f88"
Jul 17 01:53:36.631923 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:36.631802    2582 scope.go:117] "RemoveContainer" containerID="1247da1b5aa1bce0ced4dadc584183204521b60c6bdbf72cb7434b06d0c5913c"
Jul 17 01:53:36.632051 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:53:36.632021    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8_odh-ai-gateway-infra(133d1527-3ae0-4fbc-9195-8e3809969c91)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91"
Jul 17 01:53:36.647462 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:36.647439    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"]
Jul 17 01:53:37.777335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:37.777302    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"
Jul 17 01:53:37.863442 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:37.863402    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/133d1527-3ae0-4fbc-9195-8e3809969c91-maas-api-service-ca\") pod \"133d1527-3ae0-4fbc-9195-8e3809969c91\" (UID: \"133d1527-3ae0-4fbc-9195-8e3809969c91\") "
Jul 17 01:53:37.863778 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:37.863754    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/133d1527-3ae0-4fbc-9195-8e3809969c91-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "133d1527-3ae0-4fbc-9195-8e3809969c91" (UID: "133d1527-3ae0-4fbc-9195-8e3809969c91"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:53:37.964962 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:37.964931    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/133d1527-3ae0-4fbc-9195-8e3809969c91-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:53:38.643750 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:38.643710    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8" event={"ID":"133d1527-3ae0-4fbc-9195-8e3809969c91","Type":"ContainerDied","Data":"153694e7976b735573c7c167363c3dcb458a900b7a94a3aacb6c678af26f3f5e"}
Jul 17 01:53:38.643941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:38.643769    2582 scope.go:117] "RemoveContainer" containerID="1247da1b5aa1bce0ced4dadc584183204521b60c6bdbf72cb7434b06d0c5913c"
Jul 17 01:53:38.643941 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:38.643725    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"
Jul 17 01:53:38.664649 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:38.664626    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"]
Jul 17 01:53:38.669100 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:38.669079    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-kfqh8"]
Jul 17 01:53:38.990701 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:53:38.990668    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" path="/var/lib/kubelet/pods/133d1527-3ae0-4fbc-9195-8e3809969c91/volumes"
Jul 17 01:54:35.988138 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988101    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"]
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988439    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988450    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="597450fa-5820-4e80-a9ec-ecd14836d36a" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988459    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988464    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988474    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988479    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988508    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988515    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988575    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988581    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.988623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.988586    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="133d1527-3ae0-4fbc-9195-8e3809969c91" containerName="revoke-keys"
Jul 17 01:54:35.991583 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.991558    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"
Jul 17 01:54:35.993970 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.993942    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:54:35.994104 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.993942    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:54:35.998076 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:35.998052    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"]
Jul 17 01:54:36.061040 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:36.060986    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b58a97-b3b6-4b80-9ae7-1b734d25cd47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78\" (UID: \"39b58a97-b3b6-4b80-9ae7-1b734d25cd47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"
Jul 17 01:54:36.162247 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:36.162204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b58a97-b3b6-4b80-9ae7-1b734d25cd47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78\" (UID: \"39b58a97-b3b6-4b80-9ae7-1b734d25cd47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"
Jul 17 01:54:36.162826 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:36.162807    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b58a97-b3b6-4b80-9ae7-1b734d25cd47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78\" (UID: \"39b58a97-b3b6-4b80-9ae7-1b734d25cd47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"
Jul 17 01:54:36.303550 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:36.303439    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"
Jul 17 01:54:36.437911 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:36.437880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"]
Jul 17 01:54:36.441563 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:54:36.441533    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b58a97_b3b6_4b80_9ae7_1b734d25cd47.slice/crio-5f980e3c27c142a3c38bb6c87a9bbb4b71fb1b43aedbf7ab1a5f59465562553a WatchSource:0}: Error finding container 5f980e3c27c142a3c38bb6c87a9bbb4b71fb1b43aedbf7ab1a5f59465562553a: Status 404 returned error can't find the container with id 5f980e3c27c142a3c38bb6c87a9bbb4b71fb1b43aedbf7ab1a5f59465562553a
Jul 17 01:54:36.913881 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:36.913788    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" event={"ID":"39b58a97-b3b6-4b80-9ae7-1b734d25cd47","Type":"ContainerStarted","Data":"c1774953289c13d3352ea02e1e00e64a262352529f3dd6995a6ce3fe584e96e3"}
Jul 17 01:54:36.913881 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:36.913848    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" event={"ID":"39b58a97-b3b6-4b80-9ae7-1b734d25cd47","Type":"ContainerStarted","Data":"5f980e3c27c142a3c38bb6c87a9bbb4b71fb1b43aedbf7ab1a5f59465562553a"}
Jul 17 01:54:36.928100 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:36.928045    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" podStartSLOduration=1.928028897 podStartE2EDuration="1.928028897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:54:36.927349533 +0000 UTC m=+2484.431035396" watchObservedRunningTime="2026-07-17 01:54:36.928028897 +0000 UTC m=+2484.431714760"
Jul 17 01:54:37.920174 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:37.920135    2582 generic.go:356] "Generic (PLEG): container finished" podID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerID="c1774953289c13d3352ea02e1e00e64a262352529f3dd6995a6ce3fe584e96e3" exitCode=6
Jul 17 01:54:37.920598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:37.920226    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" event={"ID":"39b58a97-b3b6-4b80-9ae7-1b734d25cd47","Type":"ContainerDied","Data":"c1774953289c13d3352ea02e1e00e64a262352529f3dd6995a6ce3fe584e96e3"}
Jul 17 01:54:37.920598 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:37.920440    2582 scope.go:117] "RemoveContainer" containerID="c1774953289c13d3352ea02e1e00e64a262352529f3dd6995a6ce3fe584e96e3"
Jul 17 01:54:38.926576 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:38.926544    2582 generic.go:356] "Generic (PLEG): container finished" podID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerID="e64866a4f2408940551a2830452392ffd02d3cb5d362e62c16ae58cd1c188314" exitCode=6
Jul 17 01:54:38.927038 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:38.926631    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" event={"ID":"39b58a97-b3b6-4b80-9ae7-1b734d25cd47","Type":"ContainerDied","Data":"e64866a4f2408940551a2830452392ffd02d3cb5d362e62c16ae58cd1c188314"}
Jul 17 01:54:38.927038 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:38.926704    2582 scope.go:117] "RemoveContainer" containerID="c1774953289c13d3352ea02e1e00e64a262352529f3dd6995a6ce3fe584e96e3"
Jul 17 01:54:38.927038 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:38.926915    2582 scope.go:117] "RemoveContainer" containerID="e64866a4f2408940551a2830452392ffd02d3cb5d362e62c16ae58cd1c188314"
Jul 17 01:54:38.927205 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:54:38.927139    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78_odh-ai-gateway-infra(39b58a97-b3b6-4b80-9ae7-1b734d25cd47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47"
Jul 17 01:54:39.933870 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:39.933845    2582 scope.go:117] "RemoveContainer" containerID="e64866a4f2408940551a2830452392ffd02d3cb5d362e62c16ae58cd1c188314"
Jul 17 01:54:39.934270 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:54:39.934065    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78_odh-ai-gateway-infra(39b58a97-b3b6-4b80-9ae7-1b734d25cd47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47"
Jul 17 01:54:47.456151 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:47.456118    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:54:54.985345 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:54.985309    2582 scope.go:117] "RemoveContainer" containerID="e64866a4f2408940551a2830452392ffd02d3cb5d362e62c16ae58cd1c188314"
Jul 17 01:54:56.017629 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:56.017590    2582 generic.go:356] "Generic (PLEG): container finished" podID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerID="1651ad064c83f05c9b26dd800068c35a25100f26163b94805c64f10cf65b8060" exitCode=6
Jul 17 01:54:56.018094 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:56.017633    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" event={"ID":"39b58a97-b3b6-4b80-9ae7-1b734d25cd47","Type":"ContainerDied","Data":"1651ad064c83f05c9b26dd800068c35a25100f26163b94805c64f10cf65b8060"}
Jul 17 01:54:56.018094 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:56.017692    2582 scope.go:117] "RemoveContainer" containerID="e64866a4f2408940551a2830452392ffd02d3cb5d362e62c16ae58cd1c188314"
Jul 17 01:54:56.018094 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:56.017946    2582 scope.go:117] "RemoveContainer" containerID="1651ad064c83f05c9b26dd800068c35a25100f26163b94805c64f10cf65b8060"
Jul 17 01:54:56.018203 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:54:56.018178    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78_odh-ai-gateway-infra(39b58a97-b3b6-4b80-9ae7-1b734d25cd47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47"
Jul 17 01:54:56.361373 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:56.361293    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:54:57.041274 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:57.041240    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"]
Jul 17 01:54:57.183257 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:57.183235    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"
Jul 17 01:54:57.241370 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:57.241341    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b58a97-b3b6-4b80-9ae7-1b734d25cd47-maas-api-service-ca\") pod \"39b58a97-b3b6-4b80-9ae7-1b734d25cd47\" (UID: \"39b58a97-b3b6-4b80-9ae7-1b734d25cd47\") "
Jul 17 01:54:57.241716 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:57.241693    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39b58a97-b3b6-4b80-9ae7-1b734d25cd47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "39b58a97-b3b6-4b80-9ae7-1b734d25cd47" (UID: "39b58a97-b3b6-4b80-9ae7-1b734d25cd47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:54:57.342826 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:57.342732    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/39b58a97-b3b6-4b80-9ae7-1b734d25cd47-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:54:58.028771 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:58.028737    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78" event={"ID":"39b58a97-b3b6-4b80-9ae7-1b734d25cd47","Type":"ContainerDied","Data":"5f980e3c27c142a3c38bb6c87a9bbb4b71fb1b43aedbf7ab1a5f59465562553a"}
Jul 17 01:54:58.028771 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:58.028773    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"
Jul 17 01:54:58.029060 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:58.028799    2582 scope.go:117] "RemoveContainer" containerID="1651ad064c83f05c9b26dd800068c35a25100f26163b94805c64f10cf65b8060"
Jul 17 01:54:58.048998 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:58.048975    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"]
Jul 17 01:54:58.054881 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:58.054860    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2ef2c65d-tkb78"]
Jul 17 01:54:58.995886 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:54:58.995853    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" path="/var/lib/kubelet/pods/39b58a97-b3b6-4b80-9ae7-1b734d25cd47/volumes"
Jul 17 01:55:05.258837 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:05.258753    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:55:09.950006 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.949971    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"]
Jul 17 01:55:09.950457 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.950308    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:09.950457 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.950319    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:09.950457 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.950327    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:09.950457 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.950335    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:09.950457 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.950407    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:09.950457 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.950415    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:09.954638 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.954621    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"
Jul 17 01:55:09.956765 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.956743    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:55:09.956871 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.956797    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:55:09.960233 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:09.960213    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"]
Jul 17 01:55:10.056043 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:10.056002    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37e17c87-1454-4493-bc9a-55b7af2e81c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq\" (UID: \"37e17c87-1454-4493-bc9a-55b7af2e81c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"
Jul 17 01:55:10.157029 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:10.156997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37e17c87-1454-4493-bc9a-55b7af2e81c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq\" (UID: \"37e17c87-1454-4493-bc9a-55b7af2e81c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"
Jul 17 01:55:10.157655 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:10.157638    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37e17c87-1454-4493-bc9a-55b7af2e81c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq\" (UID: \"37e17c87-1454-4493-bc9a-55b7af2e81c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"
Jul 17 01:55:10.266835 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:10.266754    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"
Jul 17 01:55:10.411569 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:10.411542    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"]
Jul 17 01:55:10.414180 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:55:10.414150    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e17c87_1454_4493_bc9a_55b7af2e81c3.slice/crio-4790d96a2b8fcc996e3e03da21f6f534bf3ab42d335a288e702277c2fbd3c4ca WatchSource:0}: Error finding container 4790d96a2b8fcc996e3e03da21f6f534bf3ab42d335a288e702277c2fbd3c4ca: Status 404 returned error can't find the container with id 4790d96a2b8fcc996e3e03da21f6f534bf3ab42d335a288e702277c2fbd3c4ca
Jul 17 01:55:11.091534 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:11.091475    2582 generic.go:356] "Generic (PLEG): container finished" podID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerID="dec649086cdbcbee7bb5c446635eebd8734ff16d94921a6e1a8622a4882bd845" exitCode=6
Jul 17 01:55:11.091907 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:11.091570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq" event={"ID":"37e17c87-1454-4493-bc9a-55b7af2e81c3","Type":"ContainerDied","Data":"dec649086cdbcbee7bb5c446635eebd8734ff16d94921a6e1a8622a4882bd845"}
Jul 17 01:55:11.091907 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:11.091624    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq" event={"ID":"37e17c87-1454-4493-bc9a-55b7af2e81c3","Type":"ContainerStarted","Data":"4790d96a2b8fcc996e3e03da21f6f534bf3ab42d335a288e702277c2fbd3c4ca"}
Jul 17 01:55:11.091907 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:11.091805    2582 scope.go:117] "RemoveContainer" containerID="dec649086cdbcbee7bb5c446635eebd8734ff16d94921a6e1a8622a4882bd845"
Jul 17 01:55:12.097856 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:12.097768    2582 generic.go:356] "Generic (PLEG): container finished" podID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerID="943f12f0ce33b2bd50bbb0ef79568ef35e48e2b93bdc87b5f9f156bf6761d4c9" exitCode=6
Jul 17 01:55:12.098244 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:12.097846    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq" event={"ID":"37e17c87-1454-4493-bc9a-55b7af2e81c3","Type":"ContainerDied","Data":"943f12f0ce33b2bd50bbb0ef79568ef35e48e2b93bdc87b5f9f156bf6761d4c9"}
Jul 17 01:55:12.098244 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:12.097902    2582 scope.go:117] "RemoveContainer" containerID="dec649086cdbcbee7bb5c446635eebd8734ff16d94921a6e1a8622a4882bd845"
Jul 17 01:55:12.098244 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:12.098115    2582 scope.go:117] "RemoveContainer" containerID="943f12f0ce33b2bd50bbb0ef79568ef35e48e2b93bdc87b5f9f156bf6761d4c9"
Jul 17 01:55:12.098360 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:55:12.098325    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq_odh-ai-gateway-infra(37e17c87-1454-4493-bc9a-55b7af2e81c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3"
Jul 17 01:55:13.104927 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:13.104901    2582 scope.go:117] "RemoveContainer" containerID="943f12f0ce33b2bd50bbb0ef79568ef35e48e2b93bdc87b5f9f156bf6761d4c9"
Jul 17 01:55:13.105295 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:55:13.105125    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq_odh-ai-gateway-infra(37e17c87-1454-4493-bc9a-55b7af2e81c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3"
Jul 17 01:55:18.059692 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.059653    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"]
Jul 17 01:55:18.060098 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.060084    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:18.060143 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.060099    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:18.060197 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.060187    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b58a97-b3b6-4b80-9ae7-1b734d25cd47" containerName="revoke-keys"
Jul 17 01:55:18.064690 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.064673    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"
Jul 17 01:55:18.070578 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.070555    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"]
Jul 17 01:55:18.230692 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.230656    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32d3df3-6edf-4910-b0de-d3305a143eea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5\" (UID: \"d32d3df3-6edf-4910-b0de-d3305a143eea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"
Jul 17 01:55:18.331653 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.331569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32d3df3-6edf-4910-b0de-d3305a143eea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5\" (UID: \"d32d3df3-6edf-4910-b0de-d3305a143eea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"
Jul 17 01:55:18.332229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.332209    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32d3df3-6edf-4910-b0de-d3305a143eea-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5\" (UID: \"d32d3df3-6edf-4910-b0de-d3305a143eea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"
Jul 17 01:55:18.376281 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.376259    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"
Jul 17 01:55:18.512220 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:18.512191    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"]
Jul 17 01:55:18.514396 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:55:18.514371    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32d3df3_6edf_4910_b0de_d3305a143eea.slice/crio-15a25278a1cf34430a75ddc8a98323228848a48d30102b3a9bf0c2f66e364a07 WatchSource:0}: Error finding container 15a25278a1cf34430a75ddc8a98323228848a48d30102b3a9bf0c2f66e364a07: Status 404 returned error can't find the container with id 15a25278a1cf34430a75ddc8a98323228848a48d30102b3a9bf0c2f66e364a07
Jul 17 01:55:19.138745 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:19.138661    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" event={"ID":"d32d3df3-6edf-4910-b0de-d3305a143eea","Type":"ContainerStarted","Data":"61ed43043ae484665ced28bc7d110c9590c6ea29b88376f5f6f5146776e25f0d"}
Jul 17 01:55:19.138745 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:19.138728    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" event={"ID":"d32d3df3-6edf-4910-b0de-d3305a143eea","Type":"ContainerStarted","Data":"15a25278a1cf34430a75ddc8a98323228848a48d30102b3a9bf0c2f66e364a07"}
Jul 17 01:55:19.153265 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:19.153213    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" podStartSLOduration=1.15319749 podStartE2EDuration="1.15319749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:55:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:55:19.152960353 +0000 UTC m=+2526.656646216" watchObservedRunningTime="2026-07-17 01:55:19.15319749 +0000 UTC m=+2526.656883352"
Jul 17 01:55:20.144677 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:20.144643    2582 generic.go:356] "Generic (PLEG): container finished" podID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerID="61ed43043ae484665ced28bc7d110c9590c6ea29b88376f5f6f5146776e25f0d" exitCode=6
Jul 17 01:55:20.145193 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:20.144702    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" event={"ID":"d32d3df3-6edf-4910-b0de-d3305a143eea","Type":"ContainerDied","Data":"61ed43043ae484665ced28bc7d110c9590c6ea29b88376f5f6f5146776e25f0d"}
Jul 17 01:55:20.145193 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:20.144959    2582 scope.go:117] "RemoveContainer" containerID="61ed43043ae484665ced28bc7d110c9590c6ea29b88376f5f6f5146776e25f0d"
Jul 17 01:55:21.152001 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:21.151917    2582 generic.go:356] "Generic (PLEG): container finished" podID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerID="d46ceae563b829ed116130d9703336979412d71f99867ebe30f42190e7e01b7d" exitCode=6
Jul 17 01:55:21.152426 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:21.152010    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" event={"ID":"d32d3df3-6edf-4910-b0de-d3305a143eea","Type":"ContainerDied","Data":"d46ceae563b829ed116130d9703336979412d71f99867ebe30f42190e7e01b7d"}
Jul 17 01:55:21.152426 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:21.152068    2582 scope.go:117] "RemoveContainer" containerID="61ed43043ae484665ced28bc7d110c9590c6ea29b88376f5f6f5146776e25f0d"
Jul 17 01:55:21.152426 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:21.152186    2582 scope.go:117] "RemoveContainer" containerID="d46ceae563b829ed116130d9703336979412d71f99867ebe30f42190e7e01b7d"
Jul 17 01:55:21.152426 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:55:21.152390    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5_odh-ai-gateway-infra(d32d3df3-6edf-4910-b0de-d3305a143eea)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea"
Jul 17 01:55:22.157955 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:22.157928    2582 scope.go:117] "RemoveContainer" containerID="d46ceae563b829ed116130d9703336979412d71f99867ebe30f42190e7e01b7d"
Jul 17 01:55:22.158340 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:55:22.158107    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5_odh-ai-gateway-infra(d32d3df3-6edf-4910-b0de-d3305a143eea)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea"
Jul 17 01:55:23.985301 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:23.985269    2582 scope.go:117] "RemoveContainer" containerID="943f12f0ce33b2bd50bbb0ef79568ef35e48e2b93bdc87b5f9f156bf6761d4c9"
Jul 17 01:55:25.175341 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:25.175307    2582 generic.go:356] "Generic (PLEG): container finished" podID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerID="978c45fe6c7cb4d97574467163c53ba9bc4a430fb80c0f7cd387416f8c9facc7" exitCode=6
Jul 17 01:55:25.175772 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:25.175398    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq" event={"ID":"37e17c87-1454-4493-bc9a-55b7af2e81c3","Type":"ContainerDied","Data":"978c45fe6c7cb4d97574467163c53ba9bc4a430fb80c0f7cd387416f8c9facc7"}
Jul 17 01:55:25.175772 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:25.175480    2582 scope.go:117] "RemoveContainer" containerID="943f12f0ce33b2bd50bbb0ef79568ef35e48e2b93bdc87b5f9f156bf6761d4c9"
Jul 17 01:55:25.175883 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:25.175843    2582 scope.go:117] "RemoveContainer" containerID="978c45fe6c7cb4d97574467163c53ba9bc4a430fb80c0f7cd387416f8c9facc7"
Jul 17 01:55:25.176090 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:55:25.176067    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq_odh-ai-gateway-infra(37e17c87-1454-4493-bc9a-55b7af2e81c3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3"
Jul 17 01:55:26.199258 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:26.199229    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"]
Jul 17 01:55:26.323362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:26.323334    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"
Jul 17 01:55:26.509479 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:26.509398    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37e17c87-1454-4493-bc9a-55b7af2e81c3-maas-api-service-ca\") pod \"37e17c87-1454-4493-bc9a-55b7af2e81c3\" (UID: \"37e17c87-1454-4493-bc9a-55b7af2e81c3\") "
Jul 17 01:55:26.509803 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:26.509781    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37e17c87-1454-4493-bc9a-55b7af2e81c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37e17c87-1454-4493-bc9a-55b7af2e81c3" (UID: "37e17c87-1454-4493-bc9a-55b7af2e81c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:55:26.610998 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:26.610964    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37e17c87-1454-4493-bc9a-55b7af2e81c3-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:55:27.190393 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:27.190355    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq" event={"ID":"37e17c87-1454-4493-bc9a-55b7af2e81c3","Type":"ContainerDied","Data":"4790d96a2b8fcc996e3e03da21f6f534bf3ab42d335a288e702277c2fbd3c4ca"}
Jul 17 01:55:27.190393 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:27.190386    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"
Jul 17 01:55:27.190681 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:27.190433    2582 scope.go:117] "RemoveContainer" containerID="978c45fe6c7cb4d97574467163c53ba9bc4a430fb80c0f7cd387416f8c9facc7"
Jul 17 01:55:27.205726 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:27.205703    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"]
Jul 17 01:55:27.211249 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:27.211230    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-2mfqq"]
Jul 17 01:55:27.565276 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:27.565237    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:55:28.990482 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:28.990447    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" path="/var/lib/kubelet/pods/37e17c87-1454-4493-bc9a-55b7af2e81c3/volumes"
Jul 17 01:55:34.984669 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:34.984626    2582 scope.go:117] "RemoveContainer" containerID="d46ceae563b829ed116130d9703336979412d71f99867ebe30f42190e7e01b7d"
Jul 17 01:55:36.232996 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:36.232956    2582 generic.go:356] "Generic (PLEG): container finished" podID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerID="948c1f9eef95cf1b2420668259c2369156a7fec59e14d3ab95199bb1e10f681e" exitCode=6
Jul 17 01:55:36.233477 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:36.233031    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" event={"ID":"d32d3df3-6edf-4910-b0de-d3305a143eea","Type":"ContainerDied","Data":"948c1f9eef95cf1b2420668259c2369156a7fec59e14d3ab95199bb1e10f681e"}
Jul 17 01:55:36.233477 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:36.233090    2582 scope.go:117] "RemoveContainer" containerID="d46ceae563b829ed116130d9703336979412d71f99867ebe30f42190e7e01b7d"
Jul 17 01:55:36.233477 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:36.233411    2582 scope.go:117] "RemoveContainer" containerID="948c1f9eef95cf1b2420668259c2369156a7fec59e14d3ab95199bb1e10f681e"
Jul 17 01:55:36.233707 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:55:36.233687    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5_odh-ai-gateway-infra(d32d3df3-6edf-4910-b0de-d3305a143eea)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea"
Jul 17 01:55:37.256370 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:37.256343    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"]
Jul 17 01:55:37.395032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:37.395009    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"
Jul 17 01:55:37.490766 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:37.490735    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32d3df3-6edf-4910-b0de-d3305a143eea-maas-api-service-ca\") pod \"d32d3df3-6edf-4910-b0de-d3305a143eea\" (UID: \"d32d3df3-6edf-4910-b0de-d3305a143eea\") "
Jul 17 01:55:37.491138 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:37.491109    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d32d3df3-6edf-4910-b0de-d3305a143eea-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d32d3df3-6edf-4910-b0de-d3305a143eea" (UID: "d32d3df3-6edf-4910-b0de-d3305a143eea"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:55:37.592131 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:37.592035    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d32d3df3-6edf-4910-b0de-d3305a143eea-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:55:38.245932 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:38.245891    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"
Jul 17 01:55:38.245932 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:38.245908    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5" event={"ID":"d32d3df3-6edf-4910-b0de-d3305a143eea","Type":"ContainerDied","Data":"15a25278a1cf34430a75ddc8a98323228848a48d30102b3a9bf0c2f66e364a07"}
Jul 17 01:55:38.246188 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:38.245978    2582 scope.go:117] "RemoveContainer" containerID="948c1f9eef95cf1b2420668259c2369156a7fec59e14d3ab95199bb1e10f681e"
Jul 17 01:55:38.267681 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:38.267653    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"]
Jul 17 01:55:38.274370 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:38.274345    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-krmg5"]
Jul 17 01:55:38.991125 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:55:38.991090    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" path="/var/lib/kubelet/pods/d32d3df3-6edf-4910-b0de-d3305a143eea/volumes"
Jul 17 01:56:04.762553 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:04.762510    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:56:18.103010 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.102978    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"]
Jul 17 01:56:18.103436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103350    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.103436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103363    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.103436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103380    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.103436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103385    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.103436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103398    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.103436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103403    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.103436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103419    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.103436 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103425    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.103721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103483    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.103721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103507    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.103721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103512    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.103721 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.103518    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.107770 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.107753    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"
Jul 17 01:56:18.110794 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.110767    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:56:18.111218 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.111190    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:56:18.113686 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.113658    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"]
Jul 17 01:56:18.204410 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.204374    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"]
Jul 17 01:56:18.204786 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.204773    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.204849 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.204787    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.204849 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.204801    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.204849 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.204807    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.204956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.204912    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="37e17c87-1454-4493-bc9a-55b7af2e81c3" containerName="revoke-keys"
Jul 17 01:56:18.204956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.204923    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d32d3df3-6edf-4910-b0de-d3305a143eea" containerName="revoke-keys"
Jul 17 01:56:18.208073 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.208050    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.210557 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.210535    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8c5b7a14\"/\"e2e-lifecycle-model-8c5b7a14-kserve-self-signed-certs\""
Jul 17 01:56:18.210557 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.210548    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8c5b7a14\"/\"openshift-service-ca.crt\""
Jul 17 01:56:18.210739 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.210587    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8c5b7a14\"/\"kube-root-ca.crt\""
Jul 17 01:56:18.210739 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.210664    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8c5b7a14\"/\"default-dockercfg-zb98c\""
Jul 17 01:56:18.216235 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.216208    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"]
Jul 17 01:56:18.264669 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.264634    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ed0741-190d-4511-84f0-b9eb9582b878-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5\" (UID: \"a7ed0741-190d-4511-84f0-b9eb9582b878\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"
Jul 17 01:56:18.365999 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.365911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ed0741-190d-4511-84f0-b9eb9582b878-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5\" (UID: \"a7ed0741-190d-4511-84f0-b9eb9582b878\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"
Jul 17 01:56:18.365999 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.365953    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxgv2\" (UniqueName: \"kubernetes.io/projected/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-kube-api-access-wxgv2\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.366192 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.366088    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-model-cache\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.366192 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.366164    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-home\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.366192 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.366187    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-dshm\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.366295 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.366218    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tmp-dir\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.366365 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.366344    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tls-certs\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.366667 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.366652    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ed0741-190d-4511-84f0-b9eb9582b878-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5\" (UID: \"a7ed0741-190d-4511-84f0-b9eb9582b878\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"
Jul 17 01:56:18.420600 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.420566    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"
Jul 17 01:56:18.467419 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467375    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-model-cache\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.467622 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467440    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-home\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.467622 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467462    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-dshm\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.467622 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tmp-dir\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.467622 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467570    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tls-certs\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.467622 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467599    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxgv2\" (UniqueName: \"kubernetes.io/projected/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-kube-api-access-wxgv2\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.467892 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-model-cache\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.467950 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-home\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.468014 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.467982    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tmp-dir\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.470075 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.470043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-dshm\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.470561 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.470530    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tls-certs\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.476176 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.476153    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxgv2\" (UniqueName: \"kubernetes.io/projected/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-kube-api-access-wxgv2\") pod \"e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") " pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.519368 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.519325    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:18.563828 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.563791    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"]
Jul 17 01:56:18.566036 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:56:18.565999    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7ed0741_190d_4511_84f0_b9eb9582b878.slice/crio-1d93d4d0022ea647b433f843270658a46b4d047e5895f8d97372866041d879f6 WatchSource:0}: Error finding container 1d93d4d0022ea647b433f843270658a46b4d047e5895f8d97372866041d879f6: Status 404 returned error can't find the container with id 1d93d4d0022ea647b433f843270658a46b4d047e5895f8d97372866041d879f6
Jul 17 01:56:18.666859 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:18.666825    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"]
Jul 17 01:56:18.669526 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:56:18.669477    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7163bce_9bb2_483e_a2e0_6892d8dbbe4c.slice/crio-c639a25773d6145bf13112379c3f2d92579640ec4b1f129daafa91437afffbae WatchSource:0}: Error finding container c639a25773d6145bf13112379c3f2d92579640ec4b1f129daafa91437afffbae: Status 404 returned error can't find the container with id c639a25773d6145bf13112379c3f2d92579640ec4b1f129daafa91437afffbae
Jul 17 01:56:19.437260 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:19.437222    2582 generic.go:356] "Generic (PLEG): container finished" podID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerID="0d7c25281cf8278938e05a7e3cb8f19c3928440cfea09685c49b8e802726e65c" exitCode=6
Jul 17 01:56:19.437809 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:19.437320    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" event={"ID":"a7ed0741-190d-4511-84f0-b9eb9582b878","Type":"ContainerDied","Data":"0d7c25281cf8278938e05a7e3cb8f19c3928440cfea09685c49b8e802726e65c"}
Jul 17 01:56:19.437809 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:19.437385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" event={"ID":"a7ed0741-190d-4511-84f0-b9eb9582b878","Type":"ContainerStarted","Data":"1d93d4d0022ea647b433f843270658a46b4d047e5895f8d97372866041d879f6"}
Jul 17 01:56:19.437809 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:19.437538    2582 scope.go:117] "RemoveContainer" containerID="0d7c25281cf8278938e05a7e3cb8f19c3928440cfea09685c49b8e802726e65c"
Jul 17 01:56:19.442164 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:19.442133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq" event={"ID":"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c","Type":"ContainerStarted","Data":"e61c52a471f85b1c00ffd4411eec8697d5c6ca5afbadafba02954810e3f5f910"}
Jul 17 01:56:19.442277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:19.442171    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq" event={"ID":"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c","Type":"ContainerStarted","Data":"c639a25773d6145bf13112379c3f2d92579640ec4b1f129daafa91437afffbae"}
Jul 17 01:56:19.468645 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:19.468594    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq" podStartSLOduration=1.468578312 podStartE2EDuration="1.468578312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:56:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:56:19.467072032 +0000 UTC m=+2586.970757894" watchObservedRunningTime="2026-07-17 01:56:19.468578312 +0000 UTC m=+2586.972264174"
Jul 17 01:56:20.448392 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:20.448359    2582 generic.go:356] "Generic (PLEG): container finished" podID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerID="fd8a5e5ac454c22b4b57479055083bec2f13a3286ce8014b783f28309fd7a546" exitCode=6
Jul 17 01:56:20.448907 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:20.448446    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" event={"ID":"a7ed0741-190d-4511-84f0-b9eb9582b878","Type":"ContainerDied","Data":"fd8a5e5ac454c22b4b57479055083bec2f13a3286ce8014b783f28309fd7a546"}
Jul 17 01:56:20.448907 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:20.448529    2582 scope.go:117] "RemoveContainer" containerID="0d7c25281cf8278938e05a7e3cb8f19c3928440cfea09685c49b8e802726e65c"
Jul 17 01:56:20.448907 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:20.448650    2582 scope.go:117] "RemoveContainer" containerID="fd8a5e5ac454c22b4b57479055083bec2f13a3286ce8014b783f28309fd7a546"
Jul 17 01:56:20.448907 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:56:20.448867    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5_odh-ai-gateway-infra(a7ed0741-190d-4511-84f0-b9eb9582b878)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878"
Jul 17 01:56:21.458224 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:21.457733    2582 scope.go:117] "RemoveContainer" containerID="fd8a5e5ac454c22b4b57479055083bec2f13a3286ce8014b783f28309fd7a546"
Jul 17 01:56:21.458224 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:56:21.458077    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5_odh-ai-gateway-infra(a7ed0741-190d-4511-84f0-b9eb9582b878)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878"
Jul 17 01:56:22.557787 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:22.557749    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:56:28.519671 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:28.519631    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:28.520084 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:28.519687    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:28.532203 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:28.532173    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:29.506669 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:29.506640    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:56:31.984977 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:31.984947    2582 scope.go:117] "RemoveContainer" containerID="fd8a5e5ac454c22b4b57479055083bec2f13a3286ce8014b783f28309fd7a546"
Jul 17 01:56:32.510576 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:32.510473    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" event={"ID":"a7ed0741-190d-4511-84f0-b9eb9582b878","Type":"ContainerStarted","Data":"8ca27a94e5ffa47058ab28fa21d5c70cd972255367838ff1bff7585b00dc3899"}
Jul 17 01:56:32.524286 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:32.524240    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" podStartSLOduration=14.524219922 podStartE2EDuration="14.524219922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:56:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:56:32.52337905 +0000 UTC m=+2600.027064914" watchObservedRunningTime="2026-07-17 01:56:32.524219922 +0000 UTC m=+2600.027905786"
Jul 17 01:56:33.517086 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:33.517054    2582 generic.go:356] "Generic (PLEG): container finished" podID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerID="8ca27a94e5ffa47058ab28fa21d5c70cd972255367838ff1bff7585b00dc3899" exitCode=6
Jul 17 01:56:33.517616 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:33.517134    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" event={"ID":"a7ed0741-190d-4511-84f0-b9eb9582b878","Type":"ContainerDied","Data":"8ca27a94e5ffa47058ab28fa21d5c70cd972255367838ff1bff7585b00dc3899"}
Jul 17 01:56:33.517616 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:33.517213    2582 scope.go:117] "RemoveContainer" containerID="fd8a5e5ac454c22b4b57479055083bec2f13a3286ce8014b783f28309fd7a546"
Jul 17 01:56:33.517616 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:33.517528    2582 scope.go:117] "RemoveContainer" containerID="8ca27a94e5ffa47058ab28fa21d5c70cd972255367838ff1bff7585b00dc3899"
Jul 17 01:56:33.517752 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:56:33.517741    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5_odh-ai-gateway-infra(a7ed0741-190d-4511-84f0-b9eb9582b878)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878"
Jul 17 01:56:33.532743 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:33.532719    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"]
Jul 17 01:56:34.664126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:34.664095    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"
Jul 17 01:56:34.710789 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:34.710753    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ed0741-190d-4511-84f0-b9eb9582b878-maas-api-service-ca\") pod \"a7ed0741-190d-4511-84f0-b9eb9582b878\" (UID: \"a7ed0741-190d-4511-84f0-b9eb9582b878\") "
Jul 17 01:56:34.711117 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:34.711094    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7ed0741-190d-4511-84f0-b9eb9582b878-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a7ed0741-190d-4511-84f0-b9eb9582b878" (UID: "a7ed0741-190d-4511-84f0-b9eb9582b878"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:56:34.812255 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:34.812166    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7ed0741-190d-4511-84f0-b9eb9582b878-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:56:35.530229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:35.530181    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5" event={"ID":"a7ed0741-190d-4511-84f0-b9eb9582b878","Type":"ContainerDied","Data":"1d93d4d0022ea647b433f843270658a46b4d047e5895f8d97372866041d879f6"}
Jul 17 01:56:35.530229 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:35.530204    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"
Jul 17 01:56:35.530520 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:35.530258    2582 scope.go:117] "RemoveContainer" containerID="8ca27a94e5ffa47058ab28fa21d5c70cd972255367838ff1bff7585b00dc3899"
Jul 17 01:56:35.546327 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:35.546302    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"]
Jul 17 01:56:35.551399 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:35.551376    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-29qk5"]
Jul 17 01:56:36.990840 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:36.990806    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" path="/var/lib/kubelet/pods/a7ed0741-190d-4511-84f0-b9eb9582b878/volumes"
Jul 17 01:56:38.661898 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:38.661861    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:56:58.657397 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:56:58.657364    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:57:04.593682 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:04.593646    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"]
Jul 17 01:57:19.615717 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.615670    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq" podUID="f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" containerName="main" containerID="cri-o://e61c52a471f85b1c00ffd4411eec8697d5c6ca5afbadafba02954810e3f5f910" gracePeriod=45
Jul 17 01:57:19.741216 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.741185    2582 generic.go:356] "Generic (PLEG): container finished" podID="f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" containerID="e61c52a471f85b1c00ffd4411eec8697d5c6ca5afbadafba02954810e3f5f910" exitCode=0
Jul 17 01:57:19.741379 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.741263    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq" event={"ID":"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c","Type":"ContainerDied","Data":"e61c52a471f85b1c00ffd4411eec8697d5c6ca5afbadafba02954810e3f5f910"}
Jul 17 01:57:19.877886 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.877821    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:57:19.917410 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.917377    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxgv2\" (UniqueName: \"kubernetes.io/projected/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-kube-api-access-wxgv2\") pod \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") "
Jul 17 01:57:19.917615 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.917460    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tls-certs\") pod \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") "
Jul 17 01:57:19.917615 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.917542    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-home\") pod \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") "
Jul 17 01:57:19.917615 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.917592    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-dshm\") pod \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") "
Jul 17 01:57:19.917795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.917624    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tmp-dir\") pod \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") "
Jul 17 01:57:19.917795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.917659    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-model-cache\") pod \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\" (UID: \"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c\") "
Jul 17 01:57:19.917917 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.917800    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-home" (OuterVolumeSpecName: "home") pod "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" (UID: "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 01:57:19.918052 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.918028    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" (UID: "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 01:57:19.918302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.918065    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-home\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:57:19.918302 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.918217    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" (UID: "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 01:57:19.920016 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.919990    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" (UID: "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 01:57:19.920016 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.920002    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-kube-api-access-wxgv2" (OuterVolumeSpecName: "kube-api-access-wxgv2") pod "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" (UID: "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c"). InnerVolumeSpecName "kube-api-access-wxgv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 01:57:19.920170 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:19.920130    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-dshm" (OuterVolumeSpecName: "dshm") pod "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" (UID: "f7163bce-9bb2-483e-a2e0-6892d8dbbe4c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 01:57:20.019247 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.019213    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tmp-dir\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:57:20.019247 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.019242    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-model-cache\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:57:20.019247 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.019255    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxgv2\" (UniqueName: \"kubernetes.io/projected/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-kube-api-access-wxgv2\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:57:20.019522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.019266    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-tls-certs\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:57:20.019522 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.019277    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c-dshm\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:57:20.747893 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.747866    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"
Jul 17 01:57:20.748372 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.747905    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq" event={"ID":"f7163bce-9bb2-483e-a2e0-6892d8dbbe4c","Type":"ContainerDied","Data":"c639a25773d6145bf13112379c3f2d92579640ec4b1f129daafa91437afffbae"}
Jul 17 01:57:20.748372 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.747952    2582 scope.go:117] "RemoveContainer" containerID="e61c52a471f85b1c00ffd4411eec8697d5c6ca5afbadafba02954810e3f5f910"
Jul 17 01:57:20.776737 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.776708    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"]
Jul 17 01:57:20.778435 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.778402    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-8c5b7a14/e2e-lifecycle-model-8c5b7a14-kserve-6584f76c59-f87dq"]
Jul 17 01:57:20.991192 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:20.991157    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" path="/var/lib/kubelet/pods/f7163bce-9bb2-483e-a2e0-6892d8dbbe4c/volumes"
Jul 17 01:57:57.498257 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:57:57.498223    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:58:09.466852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:09.466775    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:58:13.576387 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576359    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"]
Jul 17 01:58:13.576795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576706    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:13.576795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576718    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:13.576795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576731    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:13.576795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576736    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:13.576795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576747    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" containerName="main"
Jul 17 01:58:13.576795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576752    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" containerName="main"
Jul 17 01:58:13.576795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576759    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:13.576795 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576764    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:13.577057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576848    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:13.577057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576859    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7163bce-9bb2-483e-a2e0-6892d8dbbe4c" containerName="main"
Jul 17 01:58:13.577057 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.576867    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:13.581265 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.581250    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"
Jul 17 01:58:13.583743 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.583721    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:58:13.583867 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.583782    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:58:13.591154 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.591128    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"]
Jul 17 01:58:13.697215 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.697181    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z\" (UID: \"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"
Jul 17 01:58:13.798633 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.798589    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z\" (UID: \"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"
Jul 17 01:58:13.799223 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.799204    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z\" (UID: \"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"
Jul 17 01:58:13.892407 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:13.892368    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"
Jul 17 01:58:14.025905 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.025882    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"]
Jul 17 01:58:14.027716 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:58:14.027687    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d4b41e_da5a_4838_9f57_19fb7dbbe9d6.slice/crio-eb26618408c91a320a1d5e41532bbc5e022c7d20a673a416fdb51a6b897fa349 WatchSource:0}: Error finding container eb26618408c91a320a1d5e41532bbc5e022c7d20a673a416fdb51a6b897fa349: Status 404 returned error can't find the container with id eb26618408c91a320a1d5e41532bbc5e022c7d20a673a416fdb51a6b897fa349
Jul 17 01:58:14.391480 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.391375    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:58:14.402032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.397762    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 01:58:14.402032 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.398173    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:58:14.404605 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.404569    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 01:58:14.992466 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.992379    2582 generic.go:356] "Generic (PLEG): container finished" podID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerID="7a0968aa9aa4f4cdb84855d845c49fe603f1960f49fc947a5954551a8684463b" exitCode=6
Jul 17 01:58:14.992466 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.992443    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z" event={"ID":"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6","Type":"ContainerDied","Data":"7a0968aa9aa4f4cdb84855d845c49fe603f1960f49fc947a5954551a8684463b"}
Jul 17 01:58:14.992900 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.992510    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z" event={"ID":"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6","Type":"ContainerStarted","Data":"eb26618408c91a320a1d5e41532bbc5e022c7d20a673a416fdb51a6b897fa349"}
Jul 17 01:58:14.992900 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:14.992625    2582 scope.go:117] "RemoveContainer" containerID="7a0968aa9aa4f4cdb84855d845c49fe603f1960f49fc947a5954551a8684463b"
Jul 17 01:58:15.999052 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:15.999017    2582 generic.go:356] "Generic (PLEG): container finished" podID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerID="3fa5ce2583a6dad607106dcc18f237af7d40873ff150ff87792a8d4e8646ca09" exitCode=6
Jul 17 01:58:15.999441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:15.999058    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z" event={"ID":"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6","Type":"ContainerDied","Data":"3fa5ce2583a6dad607106dcc18f237af7d40873ff150ff87792a8d4e8646ca09"}
Jul 17 01:58:15.999441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:15.999117    2582 scope.go:117] "RemoveContainer" containerID="7a0968aa9aa4f4cdb84855d845c49fe603f1960f49fc947a5954551a8684463b"
Jul 17 01:58:15.999441 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:15.999341    2582 scope.go:117] "RemoveContainer" containerID="3fa5ce2583a6dad607106dcc18f237af7d40873ff150ff87792a8d4e8646ca09"
Jul 17 01:58:15.999600 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:58:15.999580    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z_odh-ai-gateway-infra(a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6"
Jul 17 01:58:17.006083 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:17.006054    2582 scope.go:117] "RemoveContainer" containerID="3fa5ce2583a6dad607106dcc18f237af7d40873ff150ff87792a8d4e8646ca09"
Jul 17 01:58:17.006509 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:58:17.006254    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z_odh-ai-gateway-infra(a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6"
Jul 17 01:58:31.984710 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:31.984681    2582 scope.go:117] "RemoveContainer" containerID="3fa5ce2583a6dad607106dcc18f237af7d40873ff150ff87792a8d4e8646ca09"
Jul 17 01:58:33.079306 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:33.079269    2582 generic.go:356] "Generic (PLEG): container finished" podID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerID="c9702504cbdb2af1f325cb13f577c796924d5501422a0b542cb1c5137d6f100d" exitCode=6
Jul 17 01:58:33.079731 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:33.079342    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z" event={"ID":"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6","Type":"ContainerDied","Data":"c9702504cbdb2af1f325cb13f577c796924d5501422a0b542cb1c5137d6f100d"}
Jul 17 01:58:33.079731 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:33.079404    2582 scope.go:117] "RemoveContainer" containerID="3fa5ce2583a6dad607106dcc18f237af7d40873ff150ff87792a8d4e8646ca09"
Jul 17 01:58:33.079731 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:33.079663    2582 scope.go:117] "RemoveContainer" containerID="c9702504cbdb2af1f325cb13f577c796924d5501422a0b542cb1c5137d6f100d"
Jul 17 01:58:33.079896 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:58:33.079875    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z_odh-ai-gateway-infra(a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6"
Jul 17 01:58:34.105039 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:34.105007    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"]
Jul 17 01:58:34.242605 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:34.242579    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"
Jul 17 01:58:34.383003 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:34.382925    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6-maas-api-service-ca\") pod \"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6\" (UID: \"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6\") "
Jul 17 01:58:34.383272 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:34.383249    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" (UID: "a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:58:34.484315 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:34.484278    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:58:35.090395 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:35.090357    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z" event={"ID":"a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6","Type":"ContainerDied","Data":"eb26618408c91a320a1d5e41532bbc5e022c7d20a673a416fdb51a6b897fa349"}
Jul 17 01:58:35.090395 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:35.090385    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"
Jul 17 01:58:35.090760 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:35.090453    2582 scope.go:117] "RemoveContainer" containerID="c9702504cbdb2af1f325cb13f577c796924d5501422a0b542cb1c5137d6f100d"
Jul 17 01:58:35.106958 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:35.106932    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"]
Jul 17 01:58:35.110418 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:35.110396    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1e02ab63-q7g7z"]
Jul 17 01:58:36.992044 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:36.992012    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" path="/var/lib/kubelet/pods/a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6/volumes"
Jul 17 01:58:40.131414 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131378    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"]
Jul 17 01:58:40.131817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131721    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:40.131817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131732    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:40.131817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131759    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:40.131817 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131765    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:40.131956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131830    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:40.131956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131838    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:40.131956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131843    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:40.131956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.131849    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7ed0741-190d-4511-84f0-b9eb9582b878" containerName="revoke-keys"
Jul 17 01:58:40.136199 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.136181    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"
Jul 17 01:58:40.138653 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.138624    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 01:58:40.138771 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.138700    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 01:58:40.141800 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.141779    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"]
Jul 17 01:58:40.233347 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.233311    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-msc47\" (UID: \"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"
Jul 17 01:58:40.333832 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.333794    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-msc47\" (UID: \"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"
Jul 17 01:58:40.334386 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.334366    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-msc47\" (UID: \"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"
Jul 17 01:58:40.448074 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.448026    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"
Jul 17 01:58:40.587575 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:40.587399    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"]
Jul 17 01:58:41.119324 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:41.119234    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" event={"ID":"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e","Type":"ContainerStarted","Data":"08fb8af5d51749d80939adaf812a4e6f32e2837474119d549c7a967910c00c5e"}
Jul 17 01:58:41.119324 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:41.119303    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" event={"ID":"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e","Type":"ContainerStarted","Data":"2775a258dc2ae44d9b6aed7e9028cdfa449eb161bf62621df8098efe09ca7fa2"}
Jul 17 01:58:41.134548 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:41.134502    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" podStartSLOduration=1.134460141 podStartE2EDuration="1.134460141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:58:41.133731892 +0000 UTC m=+2728.637417757" watchObservedRunningTime="2026-07-17 01:58:41.134460141 +0000 UTC m=+2728.638146002"
Jul 17 01:58:42.125420 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:42.125380    2582 generic.go:356] "Generic (PLEG): container finished" podID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerID="08fb8af5d51749d80939adaf812a4e6f32e2837474119d549c7a967910c00c5e" exitCode=6
Jul 17 01:58:42.125623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:42.125435    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" event={"ID":"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e","Type":"ContainerDied","Data":"08fb8af5d51749d80939adaf812a4e6f32e2837474119d549c7a967910c00c5e"}
Jul 17 01:58:42.125704 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:42.125690    2582 scope.go:117] "RemoveContainer" containerID="08fb8af5d51749d80939adaf812a4e6f32e2837474119d549c7a967910c00c5e"
Jul 17 01:58:43.132362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:43.132280    2582 generic.go:356] "Generic (PLEG): container finished" podID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerID="d1355997ed82a1cf31d5606ecb70fd72fdec68209fedbb410642d3313c30659c" exitCode=6
Jul 17 01:58:43.132362 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:43.132334    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" event={"ID":"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e","Type":"ContainerDied","Data":"d1355997ed82a1cf31d5606ecb70fd72fdec68209fedbb410642d3313c30659c"}
Jul 17 01:58:43.132866 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:43.132386    2582 scope.go:117] "RemoveContainer" containerID="08fb8af5d51749d80939adaf812a4e6f32e2837474119d549c7a967910c00c5e"
Jul 17 01:58:43.132866 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:43.132530    2582 scope.go:117] "RemoveContainer" containerID="d1355997ed82a1cf31d5606ecb70fd72fdec68209fedbb410642d3313c30659c"
Jul 17 01:58:43.132866 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:58:43.132768    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-msc47_odh-ai-gateway-infra(e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e"
Jul 17 01:58:44.138968 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:44.138941    2582 scope.go:117] "RemoveContainer" containerID="d1355997ed82a1cf31d5606ecb70fd72fdec68209fedbb410642d3313c30659c"
Jul 17 01:58:44.139393 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:58:44.139130    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-msc47_odh-ai-gateway-infra(e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e"
Jul 17 01:58:54.827928 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.827897    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"]
Jul 17 01:58:54.828398 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.828253    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:54.828398 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.828263    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4d4b41e-da5a-4838-9f57-19fb7dbbe9d6" containerName="revoke-keys"
Jul 17 01:58:54.831339 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.831313    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.833789 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.833768    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ab7f72-openshift-default-dockercfg-cxwxv\""
Jul 17 01:58:54.842381 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.842353    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"]
Jul 17 01:58:54.961593 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-token\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.961791 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961670    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-credential-socket\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.961791 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961695    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d66f02-45c9-41f9-ac94-ff20da062e44-istiod-ca-cert\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.961791 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-certs\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.961977 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-socket\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.961977 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961852    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-podinfo\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.961977 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84h87\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-kube-api-access-84h87\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.961977 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961950    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-data\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:54.962143 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:54.961996    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-envoy\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.062875 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.062830    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-credential-socket\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.062875 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.062874    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d66f02-45c9-41f9-ac94-ff20da062e44-istiod-ca-cert\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.062897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-certs\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.062925    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-socket\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.062954    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-podinfo\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.062986    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84h87\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-kube-api-access-84h87\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.063044    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-data\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063126 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.063098    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-envoy\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063482 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.063159    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-token\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063482 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.063286    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-credential-socket\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063482 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.063355    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-socket\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063482 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.063411    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-certs\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063746 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.063717    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-data\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.063997 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.063979    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d66f02-45c9-41f9-ac94-ff20da062e44-istiod-ca-cert\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.065839 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.065816    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-envoy\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.066062 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.066039    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-podinfo\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.085924 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.085873    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-token\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.086675 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.086655    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84h87\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-kube-api-access-84h87\") pod \"e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") " pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.143392 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.143371    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:55.283576 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.283550    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"]
Jul 17 01:58:55.285345 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:58:55.285322    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61d66f02_45c9_41f9_ac94_ff20da062e44.slice/crio-20f089d7822faeaca37f71e79c44cee6aac71fc31a038d3097e2ce3ed3bfe522 WatchSource:0}: Error finding container 20f089d7822faeaca37f71e79c44cee6aac71fc31a038d3097e2ce3ed3bfe522: Status 404 returned error can't find the container with id 20f089d7822faeaca37f71e79c44cee6aac71fc31a038d3097e2ce3ed3bfe522
Jul 17 01:58:55.287355 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.287327    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:58:55.287406 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.287390    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:58:55.287449 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.287421    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:58:55.984629 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:55.984602    2582 scope.go:117] "RemoveContainer" containerID="d1355997ed82a1cf31d5606ecb70fd72fdec68209fedbb410642d3313c30659c"
Jul 17 01:58:56.195564 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:56.195532    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" event={"ID":"61d66f02-45c9-41f9-ac94-ff20da062e44","Type":"ContainerStarted","Data":"ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9"}
Jul 17 01:58:56.195736 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:56.195586    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" event={"ID":"61d66f02-45c9-41f9-ac94-ff20da062e44","Type":"ContainerStarted","Data":"20f089d7822faeaca37f71e79c44cee6aac71fc31a038d3097e2ce3ed3bfe522"}
Jul 17 01:58:56.214985 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:56.214920    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" podStartSLOduration=2.2148990729999998 podStartE2EDuration="2.214899073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:58:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:58:56.214252622 +0000 UTC m=+2743.717938490" watchObservedRunningTime="2026-07-17 01:58:56.214899073 +0000 UTC m=+2743.718584936"
Jul 17 01:58:57.144579 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.144540    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:57.149648 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.149621    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:57.202171 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.202138    2582 generic.go:356] "Generic (PLEG): container finished" podID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerID="10d49bf468d22b3522d36f0d6c663cfcd2c24325cb9b8fafa6f4a4cf55ecd52d" exitCode=6
Jul 17 01:58:57.202371 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.202217    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" event={"ID":"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e","Type":"ContainerDied","Data":"10d49bf468d22b3522d36f0d6c663cfcd2c24325cb9b8fafa6f4a4cf55ecd52d"}
Jul 17 01:58:57.202371 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.202286    2582 scope.go:117] "RemoveContainer" containerID="d1355997ed82a1cf31d5606ecb70fd72fdec68209fedbb410642d3313c30659c"
Jul 17 01:58:57.202716 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.202694    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:57.202855 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.202748    2582 scope.go:117] "RemoveContainer" containerID="10d49bf468d22b3522d36f0d6c663cfcd2c24325cb9b8fafa6f4a4cf55ecd52d"
Jul 17 01:58:57.202986 ip-10-0-134-254 kubenswrapper[2582]: E0717 01:58:57.202959    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-msc47_odh-ai-gateway-infra(e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e"
Jul 17 01:58:57.203736 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.203718    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:58:57.359284 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:57.359252    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:58:58.227870 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:58.227841    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"]
Jul 17 01:58:58.379614 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:58.379591    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"
Jul 17 01:58:58.494423 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:58.494345    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e-maas-api-service-ca\") pod \"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e\" (UID: \"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e\") "
Jul 17 01:58:58.494712 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:58.494692    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" (UID: "e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 01:58:58.595139 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:58.595111    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 01:58:59.214341 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:59.214308    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"
Jul 17 01:58:59.214527 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:59.214358    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47" event={"ID":"e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e","Type":"ContainerDied","Data":"2775a258dc2ae44d9b6aed7e9028cdfa449eb161bf62621df8098efe09ca7fa2"}
Jul 17 01:58:59.214527 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:59.214449    2582 scope.go:117] "RemoveContainer" containerID="10d49bf468d22b3522d36f0d6c663cfcd2c24325cb9b8fafa6f4a4cf55ecd52d"
Jul 17 01:58:59.229541 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:59.229508    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"]
Jul 17 01:58:59.231732 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:58:59.231710    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-msc47"]
Jul 17 01:59:00.991029 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:00.991000    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" path="/var/lib/kubelet/pods/e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e/volumes"
Jul 17 01:59:11.261946 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:11.261912    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:59:13.244719 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:13.244676    2582 patch_prober.go:28] interesting pod/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 01:59:13.245129 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:13.244742    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 01:59:14.287137 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:14.287105    2582 generic.go:356] "Generic (PLEG): container finished" podID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerID="ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9" exitCode=0
Jul 17 01:59:14.287623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:14.287186    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" event={"ID":"61d66f02-45c9-41f9-ac94-ff20da062e44","Type":"ContainerDied","Data":"ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9"}
Jul 17 01:59:14.287623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:14.287585    2582 scope.go:117] "RemoveContainer" containerID="ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9"
Jul 17 01:59:14.288633 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:14.288602    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:59:14.288737 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:14.288673    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:59:14.288737 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:14.288706    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 01:59:15.294956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:15.294919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" event={"ID":"61d66f02-45c9-41f9-ac94-ff20da062e44","Type":"ContainerStarted","Data":"e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0"}
Jul 17 01:59:16.144578 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:16.144532    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:59:16.144883 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:16.144859    2582 patch_prober.go:28] interesting pod/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 17 01:59:16.144942 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:16.144912    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 17 01:59:17.144027 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:17.143988    2582 patch_prober.go:28] interesting pod/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 17 01:59:17.144428 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:17.144050    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 17 01:59:18.144335 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:18.144298    2582 patch_prober.go:28] interesting pod/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 17 01:59:18.144752 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:18.144356    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 17 01:59:19.144603 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:19.144570    2582 patch_prober.go:28] interesting pod/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Jul 17 01:59:19.145029 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:19.144669    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Jul 17 01:59:20.148201 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:20.148171    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:59:20.148619 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:20.148499    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:59:20.149277 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:20.149260    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 01:59:25.165631 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:25.165591    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:59:27.860578 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.860539    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"]
Jul 17 01:59:27.861122 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.861099    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 01:59:27.861196 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.861127    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 01:59:27.861196 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.861171    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 01:59:27.861196 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.861181    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 01:59:27.861196 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.861191    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 01:59:27.861375 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.861199    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 01:59:27.861375 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.861318    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 01:59:27.861375 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.861330    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 01:59:27.864956 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.864935    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:27.868262 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.868237    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ab7f72\"/\"kube-root-ca.crt\""
Jul 17 01:59:27.868474 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.868290    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ab7f72\"/\"default-dockercfg-d6wmw\""
Jul 17 01:59:27.868474 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.868456    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ab7f72\"/\"openshift-service-ca.crt\""
Jul 17 01:59:27.868852 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.868830    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ab7f72\"/\"test-model-ab7f72-kserve-self-signed-certs\""
Jul 17 01:59:27.871130 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.871107    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"]
Jul 17 01:59:27.970374 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.970332    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-dshm\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:27.970620 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.970382    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tls-certs\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:27.970620 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.970449    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-model-cache\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:27.970620 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.970537    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-home\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:27.970620 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.970587    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tmp-dir\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:27.970815 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:27.970665    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2qwm\" (UniqueName: \"kubernetes.io/projected/9fbae4ba-22a2-41e6-a45d-ed31acff904f-kube-api-access-q2qwm\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.071623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.071574    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2qwm\" (UniqueName: \"kubernetes.io/projected/9fbae4ba-22a2-41e6-a45d-ed31acff904f-kube-api-access-q2qwm\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.071623 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.071635    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-dshm\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.071883 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.071660    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tls-certs\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.071883 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.071688    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-model-cache\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.071883 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.071718    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-home\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.071883 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.071842    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tmp-dir\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.072131 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.072109    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-home\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.072193 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.072121    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-model-cache\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.072239 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.072193    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tmp-dir\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.074196 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.074165    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-dshm\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.074432 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.074414    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tls-certs\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.079741 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.079723    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2qwm\" (UniqueName: \"kubernetes.io/projected/9fbae4ba-22a2-41e6-a45d-ed31acff904f-kube-api-access-q2qwm\") pod \"test-model-ab7f72-kserve-b5bc5c8dc-tn7js\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") " pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.176884 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.176860    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:28.312760 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.312664    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"]
Jul 17 01:59:28.315632 ip-10-0-134-254 kubenswrapper[2582]: W0717 01:59:28.315603    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fbae4ba_22a2_41e6_a45d_ed31acff904f.slice/crio-bc0387fb0cc416a54f0095083ddd2a1e2bcfbc38154fae150963e8ce1e9d8994 WatchSource:0}: Error finding container bc0387fb0cc416a54f0095083ddd2a1e2bcfbc38154fae150963e8ce1e9d8994: Status 404 returned error can't find the container with id bc0387fb0cc416a54f0095083ddd2a1e2bcfbc38154fae150963e8ce1e9d8994
Jul 17 01:59:28.365110 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:28.365080    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js" event={"ID":"9fbae4ba-22a2-41e6-a45d-ed31acff904f","Type":"ContainerStarted","Data":"bc0387fb0cc416a54f0095083ddd2a1e2bcfbc38154fae150963e8ce1e9d8994"}
Jul 17 01:59:29.371415 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:29.371376    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js" event={"ID":"9fbae4ba-22a2-41e6-a45d-ed31acff904f","Type":"ContainerStarted","Data":"232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b"}
Jul 17 01:59:29.387099 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:29.387048    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js" podStartSLOduration=2.387032446 podStartE2EDuration="2.387032446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:59:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:59:29.38623418 +0000 UTC m=+2776.889920044" watchObservedRunningTime="2026-07-17 01:59:29.387032446 +0000 UTC m=+2776.890718323"
Jul 17 01:59:38.177643 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:38.177549    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:38.177643 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:38.177609    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:38.190086 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:38.190061    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:38.424848 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:38.424812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 01:59:41.972822 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:41.972780    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 01:59:47.070664 ip-10-0-134-254 kubenswrapper[2582]: I0717 01:59:47.070635    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"]
Jul 17 02:00:00.270127 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:00.270090    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:00:02.091290 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.091234    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js" podUID="9fbae4ba-22a2-41e6-a45d-ed31acff904f" containerName="main" containerID="cri-o://232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b" gracePeriod=45
Jul 17 02:00:02.348588 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.348530    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 02:00:02.479549 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.479482    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-dshm\") pod \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") "
Jul 17 02:00:02.479549 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.479553    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tmp-dir\") pod \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") "
Jul 17 02:00:02.479811 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.479600    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tls-certs\") pod \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") "
Jul 17 02:00:02.480309 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.480051    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-home\") pod \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") "
Jul 17 02:00:02.480309 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.480084    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9fbae4ba-22a2-41e6-a45d-ed31acff904f" (UID: "9fbae4ba-22a2-41e6-a45d-ed31acff904f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:02.480309 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.480109    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2qwm\" (UniqueName: \"kubernetes.io/projected/9fbae4ba-22a2-41e6-a45d-ed31acff904f-kube-api-access-q2qwm\") pod \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") "
Jul 17 02:00:02.480309 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.480252    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-model-cache\") pod \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\" (UID: \"9fbae4ba-22a2-41e6-a45d-ed31acff904f\") "
Jul 17 02:00:02.480619 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.480341    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-home" (OuterVolumeSpecName: "home") pod "9fbae4ba-22a2-41e6-a45d-ed31acff904f" (UID: "9fbae4ba-22a2-41e6-a45d-ed31acff904f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:02.480674 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.480617    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-model-cache" (OuterVolumeSpecName: "model-cache") pod "9fbae4ba-22a2-41e6-a45d-ed31acff904f" (UID: "9fbae4ba-22a2-41e6-a45d-ed31acff904f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:02.487820 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.481480    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-model-cache\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:02.487820 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.481528    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tmp-dir\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:02.487820 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.481543    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-home\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:02.487820 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.482714    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-dshm" (OuterVolumeSpecName: "dshm") pod "9fbae4ba-22a2-41e6-a45d-ed31acff904f" (UID: "9fbae4ba-22a2-41e6-a45d-ed31acff904f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:02.487820 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.482828    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9fbae4ba-22a2-41e6-a45d-ed31acff904f" (UID: "9fbae4ba-22a2-41e6-a45d-ed31acff904f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 02:00:02.487820 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.483130    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fbae4ba-22a2-41e6-a45d-ed31acff904f-kube-api-access-q2qwm" (OuterVolumeSpecName: "kube-api-access-q2qwm") pod "9fbae4ba-22a2-41e6-a45d-ed31acff904f" (UID: "9fbae4ba-22a2-41e6-a45d-ed31acff904f"). InnerVolumeSpecName "kube-api-access-q2qwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:00:02.526365 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.526335    2582 generic.go:356] "Generic (PLEG): container finished" podID="9fbae4ba-22a2-41e6-a45d-ed31acff904f" containerID="232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b" exitCode=0
Jul 17 02:00:02.526567 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.526369    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js" event={"ID":"9fbae4ba-22a2-41e6-a45d-ed31acff904f","Type":"ContainerDied","Data":"232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b"}
Jul 17 02:00:02.526567 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.526402    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"
Jul 17 02:00:02.526567 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.526415    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js" event={"ID":"9fbae4ba-22a2-41e6-a45d-ed31acff904f","Type":"ContainerDied","Data":"bc0387fb0cc416a54f0095083ddd2a1e2bcfbc38154fae150963e8ce1e9d8994"}
Jul 17 02:00:02.526567 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.526432    2582 scope.go:117] "RemoveContainer" containerID="232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b"
Jul 17 02:00:02.535388 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.535372    2582 scope.go:117] "RemoveContainer" containerID="232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b"
Jul 17 02:00:02.535677 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:00:02.535656    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b\": container with ID starting with 232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b not found: ID does not exist" containerID="232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b"
Jul 17 02:00:02.535758 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.535689    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b"} err="failed to get container status \"232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b\": rpc error: code = NotFound desc = could not find container \"232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b\": container with ID starting with 232658b5b08e1e0ae67dfd47361c438df5ae29fdc4a7aca300a6d83e68dcee8b not found: ID does not exist"
Jul 17 02:00:02.546784 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.546758    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"]
Jul 17 02:00:02.550318 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.550297    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ab7f72/test-model-ab7f72-kserve-b5bc5c8dc-tn7js"]
Jul 17 02:00:02.582803 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.582778    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fbae4ba-22a2-41e6-a45d-ed31acff904f-tls-certs\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:02.582803 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.582806    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2qwm\" (UniqueName: \"kubernetes.io/projected/9fbae4ba-22a2-41e6-a45d-ed31acff904f-kube-api-access-q2qwm\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:02.582965 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.582819    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fbae4ba-22a2-41e6-a45d-ed31acff904f-dshm\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:02.991364 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:02.991333    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fbae4ba-22a2-41e6-a45d-ed31acff904f" path="/var/lib/kubelet/pods/9fbae4ba-22a2-41e6-a45d-ed31acff904f/volumes"
Jul 17 02:00:26.861403 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:26.861371    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:00:39.077582 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:39.077547    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"]
Jul 17 02:00:39.078070 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:39.077770    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy" containerID="cri-o://e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0" gracePeriod=30
Jul 17 02:00:44.332006 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.331982    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 02:00:44.449672 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.449633    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-token\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.449916 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.449688    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d66f02-45c9-41f9-ac94-ff20da062e44-istiod-ca-cert\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.449916 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.449756    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84h87\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-kube-api-access-84h87\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.449916 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.449804    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-certs\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.449916 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.449845    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-credential-socket\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.449916 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.449872    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-podinfo\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.450183 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.449981    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-envoy\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.450183 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.450042    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-socket\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.450183 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.450092    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-data\") pod \"61d66f02-45c9-41f9-ac94-ff20da062e44\" (UID: \"61d66f02-45c9-41f9-ac94-ff20da062e44\") "
Jul 17 02:00:44.450183 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.450144    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61d66f02-45c9-41f9-ac94-ff20da062e44-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 02:00:44.450596 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.450414    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61d66f02-45c9-41f9-ac94-ff20da062e44-istiod-ca-cert\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.450596 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.450435    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:44.450596 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.450454    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:44.450767 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.450614    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:44.451632 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.451608    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-data" (OuterVolumeSpecName: "istio-data") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:44.452578 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.452544    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:00:44.452578 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.452558    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 02:00:44.452736 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.452671    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-token" (OuterVolumeSpecName: "istio-token") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:00:44.452736 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.452699    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-kube-api-access-84h87" (OuterVolumeSpecName: "kube-api-access-84h87") pod "61d66f02-45c9-41f9-ac94-ff20da062e44" (UID: "61d66f02-45c9-41f9-ac94-ff20da062e44"). InnerVolumeSpecName "kube-api-access-84h87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:00:44.550875 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.550839    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84h87\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-kube-api-access-84h87\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.550875 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.550868    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-certs\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.550875 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.550878    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-credential-socket\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.550875 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.550887    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-podinfo\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.551168 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.550895    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-envoy\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.551168 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.550903    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-workload-socket\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.551168 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.550911    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-data\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.551168 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.550918    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61d66f02-45c9-41f9-ac94-ff20da062e44-istio-token\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:00:44.737599 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.737481    2582 generic.go:356] "Generic (PLEG): container finished" podID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerID="e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0" exitCode=0
Jul 17 02:00:44.737599 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.737569    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"
Jul 17 02:00:44.737599 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.737574    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" event={"ID":"61d66f02-45c9-41f9-ac94-ff20da062e44","Type":"ContainerDied","Data":"e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0"}
Jul 17 02:00:44.737848 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.737627    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf" event={"ID":"61d66f02-45c9-41f9-ac94-ff20da062e44","Type":"ContainerDied","Data":"20f089d7822faeaca37f71e79c44cee6aac71fc31a038d3097e2ce3ed3bfe522"}
Jul 17 02:00:44.737848 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.737657    2582 scope.go:117] "RemoveContainer" containerID="e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0"
Jul 17 02:00:44.747389 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.747368    2582 scope.go:117] "RemoveContainer" containerID="ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9"
Jul 17 02:00:44.755828 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.755808    2582 scope.go:117] "RemoveContainer" containerID="e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0"
Jul 17 02:00:44.756100 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:00:44.756081    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0\": container with ID starting with e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0 not found: ID does not exist" containerID="e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0"
Jul 17 02:00:44.756149 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.756110    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0"} err="failed to get container status \"e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0\": rpc error: code = NotFound desc = could not find container \"e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0\": container with ID starting with e588c670a2377e36a0705a41787c9397df327581c8b0ad746829fc66bdeb11b0 not found: ID does not exist"
Jul 17 02:00:44.756149 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.756129    2582 scope.go:117] "RemoveContainer" containerID="ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9"
Jul 17 02:00:44.756385 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:00:44.756360    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9\": container with ID starting with ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9 not found: ID does not exist" containerID="ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9"
Jul 17 02:00:44.756481 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.756391    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9"} err="failed to get container status \"ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9\": rpc error: code = NotFound desc = could not find container \"ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9\": container with ID starting with ef0758f427e6808b82c63ca616ce01c8bbacfa121fea1d5abc50fd3d01cfaab9 not found: ID does not exist"
Jul 17 02:00:44.758910 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.758888    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"]
Jul 17 02:00:44.762851 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.762827    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ab7f72-openshift-default-fcb6b68fc-ptrmf"]
Jul 17 02:00:44.990655 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:44.990571    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" path="/var/lib/kubelet/pods/61d66f02-45c9-41f9-ac94-ff20da062e44/volumes"
Jul 17 02:00:47.686303 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.686260    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"]
Jul 17 02:00:47.686845 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.686815    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy"
Jul 17 02:00:47.686845 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.686840    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy"
Jul 17 02:00:47.686845 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.686859    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy"
Jul 17 02:00:47.686845 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.686868    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy"
Jul 17 02:00:47.687161 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.686908    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbae4ba-22a2-41e6-a45d-ed31acff904f" containerName="main"
Jul 17 02:00:47.687161 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.686917    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbae4ba-22a2-41e6-a45d-ed31acff904f" containerName="main"
Jul 17 02:00:47.687161 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.687004    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e3a7a7-ec54-4b24-986c-f3ff7a7ff89e" containerName="revoke-keys"
Jul 17 02:00:47.687161 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.687017    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy"
Jul 17 02:00:47.687161 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.687026    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fbae4ba-22a2-41e6-a45d-ed31acff904f" containerName="main"
Jul 17 02:00:47.687161 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.687036    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="61d66f02-45c9-41f9-ac94-ff20da062e44" containerName="istio-proxy"
Jul 17 02:00:47.691939 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.691920    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.694278 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.694256    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d8cc4a-openshift-default-dockercfg-t9cd5\""
Jul 17 02:00:47.704549 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.704523    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"]
Jul 17 02:00:47.763222 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.763186    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:00:47.775960 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.775933    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-certs\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.776084 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.775975    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-socket\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.776084 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.776044    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-istio-token\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.776157 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.776113    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-credential-socket\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.776157 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.776137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e82f89-58f0-445a-b095-035d0a82b083-istio-podinfo\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.776226 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.776188    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-envoy\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.776226 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.776205    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e82f89-58f0-445a-b095-035d0a82b083-istiod-ca-cert\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.776288 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.776241    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jzng\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-kube-api-access-8jzng\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.776288 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.776277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-data\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877667 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877628    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-envoy\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877667 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e82f89-58f0-445a-b095-035d0a82b083-istiod-ca-cert\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877928 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877702    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jzng\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-kube-api-access-8jzng\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877928 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877733    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-data\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877928 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877762    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-certs\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877928 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-socket\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877928 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877835    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-istio-token\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877928 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877894    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-credential-socket\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.877928 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.877925    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e82f89-58f0-445a-b095-035d0a82b083-istio-podinfo\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.878306 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.878223    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-certs\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.878372 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.878328    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-credential-socket\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.878426 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.878394    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-data\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.878473 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.878455    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-socket\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.878550 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.878479    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e82f89-58f0-445a-b095-035d0a82b083-istiod-ca-cert\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.880294 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.880268    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-envoy\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.880899 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.880878    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e82f89-58f0-445a-b095-035d0a82b083-istio-podinfo\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.885156 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.885132    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-istio-token\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:47.885646 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:47.885622    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jzng\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-kube-api-access-8jzng\") pod \"e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") " pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:48.002063 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:48.001981    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:48.142329 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:48.142305    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"]
Jul 17 02:00:48.144556 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:00:48.144523    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e82f89_58f0_445a_b095_035d0a82b083.slice/crio-6d5bb707bcdc5bf2bec9bed4c72aae4155ac4dbcf7ab7646932d439768b4d35c WatchSource:0}: Error finding container 6d5bb707bcdc5bf2bec9bed4c72aae4155ac4dbcf7ab7646932d439768b4d35c: Status 404 returned error can't find the container with id 6d5bb707bcdc5bf2bec9bed4c72aae4155ac4dbcf7ab7646932d439768b4d35c
Jul 17 02:00:48.146561 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:48.146527    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:00:48.146617 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:48.146594    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:00:48.146658 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:48.146624    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:00:48.758958 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:48.758921    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" event={"ID":"60e82f89-58f0-445a-b095-035d0a82b083","Type":"ContainerStarted","Data":"0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b"}
Jul 17 02:00:48.759364 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:48.758981    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" event={"ID":"60e82f89-58f0-445a-b095-035d0a82b083","Type":"ContainerStarted","Data":"6d5bb707bcdc5bf2bec9bed4c72aae4155ac4dbcf7ab7646932d439768b4d35c"}
Jul 17 02:00:48.778816 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:48.778765    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" podStartSLOduration=1.778751601 podStartE2EDuration="1.778751601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:00:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:00:48.776893598 +0000 UTC m=+2856.280579459" watchObservedRunningTime="2026-07-17 02:00:48.778751601 +0000 UTC m=+2856.282437463"
Jul 17 02:00:49.002096 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:49.002070    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:49.006954 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:49.006926    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:49.764245 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:49.764212    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:00:49.765312 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:00:49.765290    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:01:03.835879 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:03.835786    2582 generic.go:356] "Generic (PLEG): container finished" podID="60e82f89-58f0-445a-b095-035d0a82b083" containerID="0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b" exitCode=0
Jul 17 02:01:03.835879 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:03.835836    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" event={"ID":"60e82f89-58f0-445a-b095-035d0a82b083","Type":"ContainerDied","Data":"0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b"}
Jul 17 02:01:03.836348 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:03.836241    2582 scope.go:117] "RemoveContainer" containerID="0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b"
Jul 17 02:01:03.837347 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:03.837319    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:01:03.837405 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:03.837390    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:01:03.837442 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:03.837428    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:01:04.071153 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:04.067373    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:01:04.764280 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:04.764243    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:01:04.842590 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:04.842555    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" event={"ID":"60e82f89-58f0-445a-b095-035d0a82b083","Type":"ContainerStarted","Data":"06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390"}
Jul 17 02:01:05.002642 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:05.002603    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:01:05.003807 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:05.003779    2582 patch_prober.go:28] interesting pod/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 17 02:01:05.003956 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:05.003849    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 17 02:01:06.002552 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:06.002509    2582 patch_prober.go:28] interesting pod/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 17 02:01:06.002954 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:06.002583    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 17 02:01:07.002631 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:07.002595    2582 patch_prober.go:28] interesting pod/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 17 02:01:07.003019 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:07.002646    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 17 02:01:08.002964 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:08.002924    2582 patch_prober.go:28] interesting pod/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 17 02:01:08.003357 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:08.002993    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 17 02:01:09.002752 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:09.002718    2582 patch_prober.go:28] interesting pod/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 17 02:01:09.002938 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:09.002770    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 17 02:01:10.007308 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:10.007274    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:01:10.007704 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:10.007580    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:01:10.008262 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:10.008245    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:01:19.467939 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:19.467897    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:01:43.066969 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:43.066929    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:01:55.075568 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:55.075531    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"]
Jul 17 02:01:55.076069 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:01:55.075750    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy" containerID="cri-o://06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390" gracePeriod=30
Jul 17 02:02:00.326400 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.326370    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:02:00.427977 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.427954    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-certs\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428138 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.427995    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jzng\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-kube-api-access-8jzng\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428138 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428026    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-credential-socket\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428138 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428068    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-socket\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428138 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428092    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-envoy\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428138 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428120    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e82f89-58f0-445a-b095-035d0a82b083-istio-podinfo\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428417 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428161    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e82f89-58f0-445a-b095-035d0a82b083-istiod-ca-cert\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428417 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428212    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-data\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428417 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428231    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:02:00.428417 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428241    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-istio-token\") pod \"60e82f89-58f0-445a-b095-035d0a82b083\" (UID: \"60e82f89-58f0-445a-b095-035d0a82b083\") "
Jul 17 02:02:00.428667 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428434    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:02:00.428667 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428448    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:02:00.428667 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428648    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e82f89-58f0-445a-b095-035d0a82b083-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 02:02:00.428816 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428677    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-certs\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:00.428816 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428696    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-credential-socket\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:00.428816 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.428709    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-workload-socket\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:00.429521 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.429473    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-data" (OuterVolumeSpecName: "istio-data") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:02:00.430555 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.430523    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-kube-api-access-8jzng" (OuterVolumeSpecName: "kube-api-access-8jzng") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "kube-api-access-8jzng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:02:00.430555 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.430533    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60e82f89-58f0-445a-b095-035d0a82b083-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 02:02:00.430751 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.430735    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-istio-token" (OuterVolumeSpecName: "istio-token") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:02:00.430835 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.430814    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60e82f89-58f0-445a-b095-035d0a82b083" (UID: "60e82f89-58f0-445a-b095-035d0a82b083"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:02:00.530116 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.530081    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-data\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:00.530116 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.530110    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-istio-token\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:00.530116 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.530120    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jzng\" (UniqueName: \"kubernetes.io/projected/60e82f89-58f0-445a-b095-035d0a82b083-kube-api-access-8jzng\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:00.530350 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.530131    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60e82f89-58f0-445a-b095-035d0a82b083-istio-envoy\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:00.530350 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.530140    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60e82f89-58f0-445a-b095-035d0a82b083-istio-podinfo\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:00.530350 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:00.530148    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60e82f89-58f0-445a-b095-035d0a82b083-istiod-ca-cert\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:01.104955 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.104916    2582 generic.go:356] "Generic (PLEG): container finished" podID="60e82f89-58f0-445a-b095-035d0a82b083" containerID="06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390" exitCode=0
Jul 17 02:02:01.105132 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.104962    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"
Jul 17 02:02:01.105132 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.104998    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" event={"ID":"60e82f89-58f0-445a-b095-035d0a82b083","Type":"ContainerDied","Data":"06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390"}
Jul 17 02:02:01.105132 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.105056    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md" event={"ID":"60e82f89-58f0-445a-b095-035d0a82b083","Type":"ContainerDied","Data":"6d5bb707bcdc5bf2bec9bed4c72aae4155ac4dbcf7ab7646932d439768b4d35c"}
Jul 17 02:02:01.105132 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.105078    2582 scope.go:117] "RemoveContainer" containerID="06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390"
Jul 17 02:02:01.114294 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.114278    2582 scope.go:117] "RemoveContainer" containerID="0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b"
Jul 17 02:02:01.122736 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.122718    2582 scope.go:117] "RemoveContainer" containerID="06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390"
Jul 17 02:02:01.122795 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.122778    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"]
Jul 17 02:02:01.123035 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:02:01.123016    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390\": container with ID starting with 06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390 not found: ID does not exist" containerID="06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390"
Jul 17 02:02:01.123075 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.123043    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390"} err="failed to get container status \"06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390\": rpc error: code = NotFound desc = could not find container \"06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390\": container with ID starting with 06744a1ac2aeac1e02dcaf5cba9d8e299606f91bf3139a8877a3da812370b390 not found: ID does not exist"
Jul 17 02:02:01.123075 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.123062    2582 scope.go:117] "RemoveContainer" containerID="0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b"
Jul 17 02:02:01.123316 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:02:01.123298    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b\": container with ID starting with 0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b not found: ID does not exist" containerID="0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b"
Jul 17 02:02:01.123378 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.123320    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b"} err="failed to get container status \"0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b\": rpc error: code = NotFound desc = could not find container \"0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b\": container with ID starting with 0d9afefe6361d810d1d35acc992b1bf1c8c601ae9a79033043a9eb6c1027539b not found: ID does not exist"
Jul 17 02:02:01.129139 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:01.129114    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d8cc4a-openshift-default-7869cd7547-tx8md"]
Jul 17 02:02:02.991112 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:02.991079    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60e82f89-58f0-445a-b095-035d0a82b083" path="/var/lib/kubelet/pods/60e82f89-58f0-445a-b095-035d0a82b083/volumes"
Jul 17 02:02:03.960616 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:03.960586    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:02:04.021152 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.021123    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"]
Jul 17 02:02:04.021528 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.021463    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy"
Jul 17 02:02:04.021528 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.021473    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy"
Jul 17 02:02:04.021617 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.021590    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy"
Jul 17 02:02:04.026030 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.026011    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.028241 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.028220    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-1f2720-openshift-default-dockercfg-hbqq4\""
Jul 17 02:02:04.040275 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.040246    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"]
Jul 17 02:02:04.061189 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061153    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-token\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.061313 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061228    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-credential-socket\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.061399 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061373    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-data\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.061447 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061430    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-socket\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.061525 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061455    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-podinfo\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.061592 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061549    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-certs\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.061653 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061610    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-225jr\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-kube-api-access-225jr\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.061708 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061647    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-envoy\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.061754 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.061726    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c27ac79b-331d-4675-9b9e-63c3f3095522-istiod-ca-cert\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.162792 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.162749    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-225jr\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-kube-api-access-225jr\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.162792 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.162797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-envoy\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163047 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.162944    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c27ac79b-331d-4675-9b9e-63c3f3095522-istiod-ca-cert\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163047 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163009    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-token\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163161 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-credential-socket\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163161 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163100    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-data\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163278 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163197    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-socket\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163278 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163236    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-podinfo\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163387 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163281    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-certs\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163506 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163469    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-data\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163623 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163599    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-socket\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163761 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163736    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-credential-socket\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163761 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163747    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c27ac79b-331d-4675-9b9e-63c3f3095522-istiod-ca-cert\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.163929 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.163910    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-certs\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.165298 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.165278    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-envoy\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.166213 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.166187    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-podinfo\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.170803 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.170785    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-token\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.172218 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.172199    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-225jr\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-kube-api-access-225jr\") pod \"e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") " pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.336241 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.336158    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:04.474110 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.474086    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"]
Jul 17 02:02:04.476179 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:02:04.476153    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27ac79b_331d_4675_9b9e_63c3f3095522.slice/crio-44cd164c34bd47a0c8b7dadfbb6bd35eba4b6845d803bf564d2f62a600f53ffd WatchSource:0}: Error finding container 44cd164c34bd47a0c8b7dadfbb6bd35eba4b6845d803bf564d2f62a600f53ffd: Status 404 returned error can't find the container with id 44cd164c34bd47a0c8b7dadfbb6bd35eba4b6845d803bf564d2f62a600f53ffd
Jul 17 02:02:04.478629 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.478600    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:02:04.478724 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.478665    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:02:04.478724 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:04.478694    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:02:05.126379 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:05.126344    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" event={"ID":"c27ac79b-331d-4675-9b9e-63c3f3095522","Type":"ContainerStarted","Data":"d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648"}
Jul 17 02:02:05.126795 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:05.126402    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" event={"ID":"c27ac79b-331d-4675-9b9e-63c3f3095522","Type":"ContainerStarted","Data":"44cd164c34bd47a0c8b7dadfbb6bd35eba4b6845d803bf564d2f62a600f53ffd"}
Jul 17 02:02:05.145526 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:05.145463    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" podStartSLOduration=2.145448213 podStartE2EDuration="2.145448213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:02:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:02:05.143843996 +0000 UTC m=+2932.647529859" watchObservedRunningTime="2026-07-17 02:02:05.145448213 +0000 UTC m=+2932.649134074"
Jul 17 02:02:05.337111 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:05.337076    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:05.342157 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:05.342131    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:06.130840 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:06.130809    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:06.131770 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:06.131752    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:08.380529 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.378970    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"]
Jul 17 02:02:08.380529 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.379799    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy"
Jul 17 02:02:08.380529 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.379821    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy"
Jul 17 02:02:08.380529 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.380018    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e82f89-58f0-445a-b095-035d0a82b083" containerName="istio-proxy"
Jul 17 02:02:08.384572 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.384545    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"
Jul 17 02:02:08.387405 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.387369    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 02:02:08.387405 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.387398    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 02:02:08.387711 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.387690    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"]
Jul 17 02:02:08.406676 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.406642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0df39224-e0a2-4fe7-bfb5-8c374ac8d924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z\" (UID: \"0df39224-e0a2-4fe7-bfb5-8c374ac8d924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"
Jul 17 02:02:08.507287 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.507255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0df39224-e0a2-4fe7-bfb5-8c374ac8d924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z\" (UID: \"0df39224-e0a2-4fe7-bfb5-8c374ac8d924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"
Jul 17 02:02:08.508075 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.508041    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0df39224-e0a2-4fe7-bfb5-8c374ac8d924-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z\" (UID: \"0df39224-e0a2-4fe7-bfb5-8c374ac8d924\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"
Jul 17 02:02:08.697822 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.697791    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"
Jul 17 02:02:08.830916 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:08.830816    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"]
Jul 17 02:02:08.835362 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:02:08.835328    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df39224_e0a2_4fe7_bfb5_8c374ac8d924.slice/crio-ccee016c46ef3381492c05699995677720bfbf0ae5c9af9c216e98887d3503ae WatchSource:0}: Error finding container ccee016c46ef3381492c05699995677720bfbf0ae5c9af9c216e98887d3503ae: Status 404 returned error can't find the container with id ccee016c46ef3381492c05699995677720bfbf0ae5c9af9c216e98887d3503ae
Jul 17 02:02:09.147137 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:09.147089    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" event={"ID":"0df39224-e0a2-4fe7-bfb5-8c374ac8d924","Type":"ContainerStarted","Data":"e87731d70d489b8e88baf94dbb76ff436e513a0a5f73f511933ba0c0c071d6a4"}
Jul 17 02:02:09.147319 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:09.147151    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" event={"ID":"0df39224-e0a2-4fe7-bfb5-8c374ac8d924","Type":"ContainerStarted","Data":"ccee016c46ef3381492c05699995677720bfbf0ae5c9af9c216e98887d3503ae"}
Jul 17 02:02:09.162461 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:09.162408    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" podStartSLOduration=1.162392748 podStartE2EDuration="1.162392748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:02:09.16063325 +0000 UTC m=+2936.664319111" watchObservedRunningTime="2026-07-17 02:02:09.162392748 +0000 UTC m=+2936.666078609"
Jul 17 02:02:10.153719 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:10.153686    2582 generic.go:356] "Generic (PLEG): container finished" podID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerID="e87731d70d489b8e88baf94dbb76ff436e513a0a5f73f511933ba0c0c071d6a4" exitCode=6
Jul 17 02:02:10.154102 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:10.153776    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" event={"ID":"0df39224-e0a2-4fe7-bfb5-8c374ac8d924","Type":"ContainerDied","Data":"e87731d70d489b8e88baf94dbb76ff436e513a0a5f73f511933ba0c0c071d6a4"}
Jul 17 02:02:10.154102 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:10.154020    2582 scope.go:117] "RemoveContainer" containerID="e87731d70d489b8e88baf94dbb76ff436e513a0a5f73f511933ba0c0c071d6a4"
Jul 17 02:02:11.160386 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:11.160295    2582 generic.go:356] "Generic (PLEG): container finished" podID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerID="52f3a3d1aa76b7a06ba3426a1cba073a88bbdc20ad0d8b2d218687d47fe60359" exitCode=6
Jul 17 02:02:11.160828 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:11.160383    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" event={"ID":"0df39224-e0a2-4fe7-bfb5-8c374ac8d924","Type":"ContainerDied","Data":"52f3a3d1aa76b7a06ba3426a1cba073a88bbdc20ad0d8b2d218687d47fe60359"}
Jul 17 02:02:11.160828 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:11.160444    2582 scope.go:117] "RemoveContainer" containerID="e87731d70d489b8e88baf94dbb76ff436e513a0a5f73f511933ba0c0c071d6a4"
Jul 17 02:02:11.160828 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:11.160649    2582 scope.go:117] "RemoveContainer" containerID="52f3a3d1aa76b7a06ba3426a1cba073a88bbdc20ad0d8b2d218687d47fe60359"
Jul 17 02:02:11.160956 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:02:11.160871    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z_odh-ai-gateway-infra(0df39224-e0a2-4fe7-bfb5-8c374ac8d924)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924"
Jul 17 02:02:12.167833 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:12.167808    2582 scope.go:117] "RemoveContainer" containerID="52f3a3d1aa76b7a06ba3426a1cba073a88bbdc20ad0d8b2d218687d47fe60359"
Jul 17 02:02:12.168224 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:02:12.167989    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z_odh-ai-gateway-infra(0df39224-e0a2-4fe7-bfb5-8c374ac8d924)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924"
Jul 17 02:02:19.206931 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:19.206893    2582 generic.go:356] "Generic (PLEG): container finished" podID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerID="d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648" exitCode=0
Jul 17 02:02:19.207418 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:19.206959    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" event={"ID":"c27ac79b-331d-4675-9b9e-63c3f3095522","Type":"ContainerDied","Data":"d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648"}
Jul 17 02:02:19.207418 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:19.207344    2582 scope.go:117] "RemoveContainer" containerID="d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648"
Jul 17 02:02:19.208475 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:19.208447    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:02:19.208581 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:19.208562    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:02:19.208641 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:19.208615    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 17 02:02:19.269077 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:19.269050    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:02:20.217270 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:20.217225    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" event={"ID":"c27ac79b-331d-4675-9b9e-63c3f3095522","Type":"ContainerStarted","Data":"fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0"}
Jul 17 02:02:20.337252 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:20.337214    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:20.338616 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:20.338584    2582 patch_prober.go:28] interesting pod/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 17 02:02:20.338760 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:20.338640    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 17 02:02:21.130802 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:21.130764    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:21.337232 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:21.337194    2582 patch_prober.go:28] interesting pod/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 17 02:02:21.337645 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:21.337257    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 17 02:02:22.337143 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:22.337098    2582 patch_prober.go:28] interesting pod/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 17 02:02:22.337318 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:22.337173    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 17 02:02:22.988794 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:22.988767    2582 scope.go:117] "RemoveContainer" containerID="52f3a3d1aa76b7a06ba3426a1cba073a88bbdc20ad0d8b2d218687d47fe60359"
Jul 17 02:02:23.337641 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:23.337611    2582 patch_prober.go:28] interesting pod/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Jul 17 02:02:23.337952 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:23.337668    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Jul 17 02:02:24.244222 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:24.244130    2582 generic.go:356] "Generic (PLEG): container finished" podID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerID="0d3712e2478e3084d2f5bc37e1f5f2349853e8c56033888230ce516d0b1cac96" exitCode=6
Jul 17 02:02:24.244375 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:24.244214    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" event={"ID":"0df39224-e0a2-4fe7-bfb5-8c374ac8d924","Type":"ContainerDied","Data":"0d3712e2478e3084d2f5bc37e1f5f2349853e8c56033888230ce516d0b1cac96"}
Jul 17 02:02:24.244375 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:24.244295    2582 scope.go:117] "RemoveContainer" containerID="52f3a3d1aa76b7a06ba3426a1cba073a88bbdc20ad0d8b2d218687d47fe60359"
Jul 17 02:02:24.244703 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:24.244679    2582 scope.go:117] "RemoveContainer" containerID="0d3712e2478e3084d2f5bc37e1f5f2349853e8c56033888230ce516d0b1cac96"
Jul 17 02:02:24.244992 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:02:24.244952    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z_odh-ai-gateway-infra(0df39224-e0a2-4fe7-bfb5-8c374ac8d924)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924"
Jul 17 02:02:24.341099 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:24.341073    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:24.342085 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:24.342062    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:02:25.268287 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:25.268255    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"]
Jul 17 02:02:25.412580 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:25.412556    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"
Jul 17 02:02:25.468925 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:25.468893    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0df39224-e0a2-4fe7-bfb5-8c374ac8d924-maas-api-service-ca\") pod \"0df39224-e0a2-4fe7-bfb5-8c374ac8d924\" (UID: \"0df39224-e0a2-4fe7-bfb5-8c374ac8d924\") "
Jul 17 02:02:25.469245 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:25.469219    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0df39224-e0a2-4fe7-bfb5-8c374ac8d924-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0df39224-e0a2-4fe7-bfb5-8c374ac8d924" (UID: "0df39224-e0a2-4fe7-bfb5-8c374ac8d924"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 02:02:25.570160 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:25.570075    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0df39224-e0a2-4fe7-bfb5-8c374ac8d924-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:02:26.257643 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:26.257616    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"
Jul 17 02:02:26.257824 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:26.257612    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z" event={"ID":"0df39224-e0a2-4fe7-bfb5-8c374ac8d924","Type":"ContainerDied","Data":"ccee016c46ef3381492c05699995677720bfbf0ae5c9af9c216e98887d3503ae"}
Jul 17 02:02:26.257824 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:26.257753    2582 scope.go:117] "RemoveContainer" containerID="0d3712e2478e3084d2f5bc37e1f5f2349853e8c56033888230ce516d0b1cac96"
Jul 17 02:02:26.279752 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:26.279728    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"]
Jul 17 02:02:26.285183 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:26.285162    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-dn46z"]
Jul 17 02:02:26.991331 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:26.991296    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" path="/var/lib/kubelet/pods/0df39224-e0a2-4fe7-bfb5-8c374ac8d924/volumes"
Jul 17 02:02:29.057240 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:29.057203    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:02:55.367936 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.367845    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"]
Jul 17 02:02:55.368541 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.368398    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:02:55.368541 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.368418    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:02:55.368541 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.368449    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:02:55.368541 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.368458    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:02:55.368541 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.368477    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:02:55.368541 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.368511    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:02:55.368892 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.368614    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:02:55.368892 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.368630    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:02:55.371834 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.371811    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.374791 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.374768    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"kube-root-ca.crt\""
Jul 17 02:02:55.375798 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.375777    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"openshift-service-ca.crt\""
Jul 17 02:02:55.375988 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.375788    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"default-dockercfg-hpl7h\""
Jul 17 02:02:55.376076 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.375790    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"auth-test-model-1f2720-kserve-self-signed-certs\""
Jul 17 02:02:55.382314 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.382286    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"]
Jul 17 02:02:55.438297 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.438261    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-tmp-dir\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.438469 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.438313    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twnww\" (UniqueName: \"kubernetes.io/projected/17816ee8-5306-42d6-b05e-32c185cbe929-kube-api-access-twnww\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.438469 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.438368    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-home\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.438469 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.438388    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17816ee8-5306-42d6-b05e-32c185cbe929-tls-certs\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.438469 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.438442    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-dshm\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.438469 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.438458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-model-cache\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.538914 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.538877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-home\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.538914 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.538918    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17816ee8-5306-42d6-b05e-32c185cbe929-tls-certs\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.539144 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.538959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-dshm\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.539144 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.538975    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-model-cache\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.539144 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.539049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-tmp-dir\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.539144 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.539101    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twnww\" (UniqueName: \"kubernetes.io/projected/17816ee8-5306-42d6-b05e-32c185cbe929-kube-api-access-twnww\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.539371 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.539349    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-home\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.539371 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.539361    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-model-cache\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.539619 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.539596    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-tmp-dir\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.541623 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.541589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-dshm\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.541787 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.541767    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17816ee8-5306-42d6-b05e-32c185cbe929-tls-certs\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.546708 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.546679    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twnww\" (UniqueName: \"kubernetes.io/projected/17816ee8-5306-42d6-b05e-32c185cbe929-kube-api-access-twnww\") pod \"auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") " pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.683536 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.683463    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:02:55.834663 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:55.834629    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"]
Jul 17 02:02:55.837595 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:02:55.837556    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17816ee8_5306_42d6_b05e_32c185cbe929.slice/crio-ef719a9aaed3b80b006c5a71f071816e82d3de834929b24dfdc11a84318b0db4 WatchSource:0}: Error finding container ef719a9aaed3b80b006c5a71f071816e82d3de834929b24dfdc11a84318b0db4: Status 404 returned error can't find the container with id ef719a9aaed3b80b006c5a71f071816e82d3de834929b24dfdc11a84318b0db4
Jul 17 02:02:56.404553 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:56.404520    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2" event={"ID":"17816ee8-5306-42d6-b05e-32c185cbe929","Type":"ContainerStarted","Data":"bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2"}
Jul 17 02:02:56.404958 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:56.404562    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2" event={"ID":"17816ee8-5306-42d6-b05e-32c185cbe929","Type":"ContainerStarted","Data":"ef719a9aaed3b80b006c5a71f071816e82d3de834929b24dfdc11a84318b0db4"}
Jul 17 02:02:56.423770 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:56.423710    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2" podStartSLOduration=1.423692285 podStartE2EDuration="1.423692285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:02:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:02:56.423131842 +0000 UTC m=+2983.926817706" watchObservedRunningTime="2026-07-17 02:02:56.423692285 +0000 UTC m=+2983.927378146"
Jul 17 02:02:59.059531 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:02:59.059479    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:03:05.684592 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:05.684558    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:03:05.685067 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:05.684608    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:03:05.697658 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:05.697629    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:03:06.466826 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:06.466797    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:03:14.420130 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:14.420106    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 02:03:14.426736 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:14.426713    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 02:03:14.427469 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:14.427448    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 02:03:14.433735 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:14.433708    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 02:03:15.462590 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:15.462555    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:03:33.767041 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:33.767001    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:03:48.460534 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:03:48.460478    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:04:05.662527 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:04:05.662420    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:04:23.073152 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:04:23.073118    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:04:39.963373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:04:39.963337    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:04:54.353792 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:04:54.353755    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"]
Jul 17 02:04:57.763918 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:04:57.763880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:05:09.375004 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.374947    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2" podUID="17816ee8-5306-42d6-b05e-32c185cbe929" containerName="main" containerID="cri-o://bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2" gracePeriod=45
Jul 17 02:05:09.633867 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.633802    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:05:09.742916 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.742882    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-model-cache\") pod \"17816ee8-5306-42d6-b05e-32c185cbe929\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") "
Jul 17 02:05:09.743096 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.742926    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17816ee8-5306-42d6-b05e-32c185cbe929-tls-certs\") pod \"17816ee8-5306-42d6-b05e-32c185cbe929\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") "
Jul 17 02:05:09.743096 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.742984    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-dshm\") pod \"17816ee8-5306-42d6-b05e-32c185cbe929\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") "
Jul 17 02:05:09.743096 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.743008    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twnww\" (UniqueName: \"kubernetes.io/projected/17816ee8-5306-42d6-b05e-32c185cbe929-kube-api-access-twnww\") pod \"17816ee8-5306-42d6-b05e-32c185cbe929\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") "
Jul 17 02:05:09.743096 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.743040    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-tmp-dir\") pod \"17816ee8-5306-42d6-b05e-32c185cbe929\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") "
Jul 17 02:05:09.743096 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.743064    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-home\") pod \"17816ee8-5306-42d6-b05e-32c185cbe929\" (UID: \"17816ee8-5306-42d6-b05e-32c185cbe929\") "
Jul 17 02:05:09.743375 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.743235    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-model-cache" (OuterVolumeSpecName: "model-cache") pod "17816ee8-5306-42d6-b05e-32c185cbe929" (UID: "17816ee8-5306-42d6-b05e-32c185cbe929"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:05:09.743375 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.743349    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-model-cache\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:05:09.743463 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.743363    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "17816ee8-5306-42d6-b05e-32c185cbe929" (UID: "17816ee8-5306-42d6-b05e-32c185cbe929"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:05:09.743463 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.743369    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-home" (OuterVolumeSpecName: "home") pod "17816ee8-5306-42d6-b05e-32c185cbe929" (UID: "17816ee8-5306-42d6-b05e-32c185cbe929"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:05:09.745528 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.745477    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17816ee8-5306-42d6-b05e-32c185cbe929-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "17816ee8-5306-42d6-b05e-32c185cbe929" (UID: "17816ee8-5306-42d6-b05e-32c185cbe929"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 02:05:09.745528 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.745471    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17816ee8-5306-42d6-b05e-32c185cbe929-kube-api-access-twnww" (OuterVolumeSpecName: "kube-api-access-twnww") pod "17816ee8-5306-42d6-b05e-32c185cbe929" (UID: "17816ee8-5306-42d6-b05e-32c185cbe929"). InnerVolumeSpecName "kube-api-access-twnww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:05:09.745718 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.745548    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-dshm" (OuterVolumeSpecName: "dshm") pod "17816ee8-5306-42d6-b05e-32c185cbe929" (UID: "17816ee8-5306-42d6-b05e-32c185cbe929"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:05:09.844711 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.844673    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-tmp-dir\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:05:09.844711 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.844703    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-home\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:05:09.844711 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.844712    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17816ee8-5306-42d6-b05e-32c185cbe929-tls-certs\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:05:09.844711 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.844721    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17816ee8-5306-42d6-b05e-32c185cbe929-dshm\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:05:09.844994 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:09.844730    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twnww\" (UniqueName: \"kubernetes.io/projected/17816ee8-5306-42d6-b05e-32c185cbe929-kube-api-access-twnww\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:05:10.043134 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.043098    2582 generic.go:356] "Generic (PLEG): container finished" podID="17816ee8-5306-42d6-b05e-32c185cbe929" containerID="bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2" exitCode=0
Jul 17 02:05:10.043381 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.043150    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"
Jul 17 02:05:10.043381 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.043168    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2" event={"ID":"17816ee8-5306-42d6-b05e-32c185cbe929","Type":"ContainerDied","Data":"bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2"}
Jul 17 02:05:10.043381 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.043223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2" event={"ID":"17816ee8-5306-42d6-b05e-32c185cbe929","Type":"ContainerDied","Data":"ef719a9aaed3b80b006c5a71f071816e82d3de834929b24dfdc11a84318b0db4"}
Jul 17 02:05:10.043381 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.043245    2582 scope.go:117] "RemoveContainer" containerID="bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2"
Jul 17 02:05:10.054235 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.054218    2582 scope.go:117] "RemoveContainer" containerID="bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2"
Jul 17 02:05:10.054534 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:05:10.054514    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2\": container with ID starting with bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2 not found: ID does not exist" containerID="bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2"
Jul 17 02:05:10.054619 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.054547    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2"} err="failed to get container status \"bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2\": rpc error: code = NotFound desc = could not find container \"bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2\": container with ID starting with bf9cea7fecb630c519df65597063707e2842f5756d84d8ecb0d25697468963d2 not found: ID does not exist"
Jul 17 02:05:10.066690 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.066657    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"]
Jul 17 02:05:10.068917 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.068892    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/auth-test-model-1f2720-kserve-5bffcbc5f-wgjj2"]
Jul 17 02:05:10.990262 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:10.990229    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17816ee8-5306-42d6-b05e-32c185cbe929" path="/var/lib/kubelet/pods/17816ee8-5306-42d6-b05e-32c185cbe929/volumes"
Jul 17 02:05:19.669016 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:19.668977    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:05:41.669578 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.669465    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"]
Jul 17 02:05:41.670223 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.670196    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17816ee8-5306-42d6-b05e-32c185cbe929" containerName="main"
Jul 17 02:05:41.670223 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.670222    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="17816ee8-5306-42d6-b05e-32c185cbe929" containerName="main"
Jul 17 02:05:41.670444 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.670318    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="0df39224-e0a2-4fe7-bfb5-8c374ac8d924" containerName="revoke-keys"
Jul 17 02:05:41.670444 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.670331    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="17816ee8-5306-42d6-b05e-32c185cbe929" containerName="main"
Jul 17 02:05:41.673376 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.673358    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.677119 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.677099    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"default-dockercfg-hpl7h\""
Jul 17 02:05:41.677254 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.677099    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"kube-root-ca.crt\""
Jul 17 02:05:41.677254 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.677158    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"openshift-service-ca.crt\""
Jul 17 02:05:41.677383 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.677100    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"sub-test-model-1f2720-kserve-self-signed-certs\""
Jul 17 02:05:41.683880 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.683845    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"]
Jul 17 02:05:41.731195 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.731161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613d3221-7df5-4675-90d4-c3a5ee768639-tls-certs\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.731353 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.731209    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-tmp-dir\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.731353 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.731260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-home\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.731353 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.731277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-dshm\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.731353 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.731294    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-model-cache\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.731523 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.731372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vzlt\" (UniqueName: \"kubernetes.io/projected/613d3221-7df5-4675-90d4-c3a5ee768639-kube-api-access-6vzlt\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832071 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832025    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-model-cache\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832249 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832105    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vzlt\" (UniqueName: \"kubernetes.io/projected/613d3221-7df5-4675-90d4-c3a5ee768639-kube-api-access-6vzlt\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832249 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832135    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613d3221-7df5-4675-90d4-c3a5ee768639-tls-certs\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832249 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-tmp-dir\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832249 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-home\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832414 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-dshm\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832451 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832415    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-model-cache\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832607 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832589    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-home\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.832649 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.832581    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-tmp-dir\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.834650 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.834629    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-dshm\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.834893 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.834879    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613d3221-7df5-4675-90d4-c3a5ee768639-tls-certs\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.839803 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.839782    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vzlt\" (UniqueName: \"kubernetes.io/projected/613d3221-7df5-4675-90d4-c3a5ee768639-kube-api-access-6vzlt\") pod \"sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") " pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:41.985085 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:41.985055    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:42.125510 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:42.125463    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"]
Jul 17 02:05:42.127462 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:05:42.127431    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod613d3221_7df5_4675_90d4_c3a5ee768639.slice/crio-76044db5de1e0bbf638eeb8976abefcba5351c99fa64cce36e4bab225c7fc80e WatchSource:0}: Error finding container 76044db5de1e0bbf638eeb8976abefcba5351c99fa64cce36e4bab225c7fc80e: Status 404 returned error can't find the container with id 76044db5de1e0bbf638eeb8976abefcba5351c99fa64cce36e4bab225c7fc80e
Jul 17 02:05:42.198538 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:42.198482    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99" event={"ID":"613d3221-7df5-4675-90d4-c3a5ee768639","Type":"ContainerStarted","Data":"224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53"}
Jul 17 02:05:42.198684 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:42.198554    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99" event={"ID":"613d3221-7df5-4675-90d4-c3a5ee768639","Type":"ContainerStarted","Data":"76044db5de1e0bbf638eeb8976abefcba5351c99fa64cce36e4bab225c7fc80e"}
Jul 17 02:05:42.215289 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:42.215230    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99" podStartSLOduration=1.215212302 podStartE2EDuration="1.215212302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:05:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:05:42.21417317 +0000 UTC m=+3149.717859034" watchObservedRunningTime="2026-07-17 02:05:42.215212302 +0000 UTC m=+3149.718898164"
Jul 17 02:05:45.956938 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:45.956902    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:05:51.985955 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:51.985923    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:51.986369 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:51.986160    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:51.998771 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:51.998743    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:05:52.261553 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:05:52.261440    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:06:02.573251 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:02.573211    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:06:21.152473 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:21.152435    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"]
Jul 17 02:06:24.961740 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:24.961705    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:06:36.173677 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.173612    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99" podUID="613d3221-7df5-4675-90d4-c3a5ee768639" containerName="main" containerID="cri-o://224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53" gracePeriod=45
Jul 17 02:06:36.440206 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.440181    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:06:36.464176 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.464140    2582 generic.go:356] "Generic (PLEG): container finished" podID="613d3221-7df5-4675-90d4-c3a5ee768639" containerID="224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53" exitCode=0
Jul 17 02:06:36.464368 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.464186    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"
Jul 17 02:06:36.464368 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.464215    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99" event={"ID":"613d3221-7df5-4675-90d4-c3a5ee768639","Type":"ContainerDied","Data":"224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53"}
Jul 17 02:06:36.464368 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.464264    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99" event={"ID":"613d3221-7df5-4675-90d4-c3a5ee768639","Type":"ContainerDied","Data":"76044db5de1e0bbf638eeb8976abefcba5351c99fa64cce36e4bab225c7fc80e"}
Jul 17 02:06:36.464368 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.464283    2582 scope.go:117] "RemoveContainer" containerID="224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53"
Jul 17 02:06:36.473412 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.473388    2582 scope.go:117] "RemoveContainer" containerID="224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53"
Jul 17 02:06:36.473759 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:06:36.473733    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53\": container with ID starting with 224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53 not found: ID does not exist" containerID="224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53"
Jul 17 02:06:36.473811 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.473763    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53"} err="failed to get container status \"224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53\": rpc error: code = NotFound desc = could not find container \"224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53\": container with ID starting with 224f618c4dcca1374f0f08a7bd7cf935ce45c05cefde4642a87cde010071aa53 not found: ID does not exist"
Jul 17 02:06:36.539854 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.539817    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613d3221-7df5-4675-90d4-c3a5ee768639-tls-certs\") pod \"613d3221-7df5-4675-90d4-c3a5ee768639\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") "
Jul 17 02:06:36.540055 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.539875    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-model-cache\") pod \"613d3221-7df5-4675-90d4-c3a5ee768639\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") "
Jul 17 02:06:36.540055 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.539903    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-tmp-dir\") pod \"613d3221-7df5-4675-90d4-c3a5ee768639\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") "
Jul 17 02:06:36.540055 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540020    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vzlt\" (UniqueName: \"kubernetes.io/projected/613d3221-7df5-4675-90d4-c3a5ee768639-kube-api-access-6vzlt\") pod \"613d3221-7df5-4675-90d4-c3a5ee768639\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") "
Jul 17 02:06:36.540243 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540072    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-home\") pod \"613d3221-7df5-4675-90d4-c3a5ee768639\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") "
Jul 17 02:06:36.540243 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540122    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-dshm\") pod \"613d3221-7df5-4675-90d4-c3a5ee768639\" (UID: \"613d3221-7df5-4675-90d4-c3a5ee768639\") "
Jul 17 02:06:36.540243 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540135    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "613d3221-7df5-4675-90d4-c3a5ee768639" (UID: "613d3221-7df5-4675-90d4-c3a5ee768639"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:06:36.540243 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540170    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-model-cache" (OuterVolumeSpecName: "model-cache") pod "613d3221-7df5-4675-90d4-c3a5ee768639" (UID: "613d3221-7df5-4675-90d4-c3a5ee768639"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:06:36.540419 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540345    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-home" (OuterVolumeSpecName: "home") pod "613d3221-7df5-4675-90d4-c3a5ee768639" (UID: "613d3221-7df5-4675-90d4-c3a5ee768639"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:06:36.540461 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540427    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-model-cache\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:06:36.540461 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540443    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-tmp-dir\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:06:36.540461 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.540452    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-home\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:06:36.542178 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.542151    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-dshm" (OuterVolumeSpecName: "dshm") pod "613d3221-7df5-4675-90d4-c3a5ee768639" (UID: "613d3221-7df5-4675-90d4-c3a5ee768639"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:06:36.542294 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.542269    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/613d3221-7df5-4675-90d4-c3a5ee768639-kube-api-access-6vzlt" (OuterVolumeSpecName: "kube-api-access-6vzlt") pod "613d3221-7df5-4675-90d4-c3a5ee768639" (UID: "613d3221-7df5-4675-90d4-c3a5ee768639"). InnerVolumeSpecName "kube-api-access-6vzlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:06:36.542339 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.542326    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/613d3221-7df5-4675-90d4-c3a5ee768639-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "613d3221-7df5-4675-90d4-c3a5ee768639" (UID: "613d3221-7df5-4675-90d4-c3a5ee768639"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 02:06:36.641359 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.641313    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613d3221-7df5-4675-90d4-c3a5ee768639-tls-certs\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:06:36.641359 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.641348    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vzlt\" (UniqueName: \"kubernetes.io/projected/613d3221-7df5-4675-90d4-c3a5ee768639-kube-api-access-6vzlt\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:06:36.641359 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.641364    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/613d3221-7df5-4675-90d4-c3a5ee768639-dshm\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:06:36.785444 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.785412    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"]
Jul 17 02:06:36.789171 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.789148    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/sub-test-model-1f2720-kserve-6dfdcf7bc5-gvj99"]
Jul 17 02:06:36.991149 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:36.991115    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="613d3221-7df5-4675-90d4-c3a5ee768639" path="/var/lib/kubelet/pods/613d3221-7df5-4675-90d4-c3a5ee768639/volumes"
Jul 17 02:06:38.579370 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.579334    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"]
Jul 17 02:06:38.579795 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.579733    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613d3221-7df5-4675-90d4-c3a5ee768639" containerName="main"
Jul 17 02:06:38.579795 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.579745    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="613d3221-7df5-4675-90d4-c3a5ee768639" containerName="main"
Jul 17 02:06:38.579876 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.579802    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="613d3221-7df5-4675-90d4-c3a5ee768639" containerName="main"
Jul 17 02:06:38.584106 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.584089    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"
Jul 17 02:06:38.586260 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.586242    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 02:06:38.586366 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.586323    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 02:06:38.589850 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.589824    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"]
Jul 17 02:06:38.659957 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.659921    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw\" (UID: \"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"
Jul 17 02:06:38.761124 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.761088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw\" (UID: \"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"
Jul 17 02:06:38.761776 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.761754    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw\" (UID: \"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"
Jul 17 02:06:38.895912 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:38.895826    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"
Jul 17 02:06:39.036102 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:39.036077    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"]
Jul 17 02:06:39.038547 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:06:39.038520    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b3b17e_4e79_4f0e_bfcb_fe6b1c5c7958.slice/crio-15ce8985ba14c5a59695422e572364983957b2842e7af05288d0d0fc3b4389e2 WatchSource:0}: Error finding container 15ce8985ba14c5a59695422e572364983957b2842e7af05288d0d0fc3b4389e2: Status 404 returned error can't find the container with id 15ce8985ba14c5a59695422e572364983957b2842e7af05288d0d0fc3b4389e2
Jul 17 02:06:39.480918 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:39.480823    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" event={"ID":"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958","Type":"ContainerStarted","Data":"c637916e54c07831906f3f4b7e456d159b25e8a4101c94702b3c3bd470966b32"}
Jul 17 02:06:39.480918 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:39.480878    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" event={"ID":"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958","Type":"ContainerStarted","Data":"15ce8985ba14c5a59695422e572364983957b2842e7af05288d0d0fc3b4389e2"}
Jul 17 02:06:39.497085 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:39.497033    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" podStartSLOduration=1.497017328 podStartE2EDuration="1.497017328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:06:39.495299259 +0000 UTC m=+3206.998985121" watchObservedRunningTime="2026-07-17 02:06:39.497017328 +0000 UTC m=+3207.000703189"
Jul 17 02:06:40.468148 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:40.468114    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:06:40.492052 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:40.492018    2582 generic.go:356] "Generic (PLEG): container finished" podID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerID="c637916e54c07831906f3f4b7e456d159b25e8a4101c94702b3c3bd470966b32" exitCode=6
Jul 17 02:06:40.492246 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:40.492108    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" event={"ID":"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958","Type":"ContainerDied","Data":"c637916e54c07831906f3f4b7e456d159b25e8a4101c94702b3c3bd470966b32"}
Jul 17 02:06:40.492291 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:40.492276    2582 scope.go:117] "RemoveContainer" containerID="c637916e54c07831906f3f4b7e456d159b25e8a4101c94702b3c3bd470966b32"
Jul 17 02:06:41.499004 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:41.498970    2582 generic.go:356] "Generic (PLEG): container finished" podID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerID="bf44a587079c9c2585f60af4fbfec6c625eee9cfd0e8ea7aee9ab1cce03d14b7" exitCode=6
Jul 17 02:06:41.499623 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:41.499041    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" event={"ID":"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958","Type":"ContainerDied","Data":"bf44a587079c9c2585f60af4fbfec6c625eee9cfd0e8ea7aee9ab1cce03d14b7"}
Jul 17 02:06:41.499623 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:41.499090    2582 scope.go:117] "RemoveContainer" containerID="c637916e54c07831906f3f4b7e456d159b25e8a4101c94702b3c3bd470966b32"
Jul 17 02:06:41.499623 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:41.499291    2582 scope.go:117] "RemoveContainer" containerID="bf44a587079c9c2585f60af4fbfec6c625eee9cfd0e8ea7aee9ab1cce03d14b7"
Jul 17 02:06:41.499623 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:06:41.499576    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw_odh-ai-gateway-infra(12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958"
Jul 17 02:06:42.506180 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:42.506156    2582 scope.go:117] "RemoveContainer" containerID="bf44a587079c9c2585f60af4fbfec6c625eee9cfd0e8ea7aee9ab1cce03d14b7"
Jul 17 02:06:42.506597 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:06:42.506333    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw_odh-ai-gateway-infra(12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958"
Jul 17 02:06:49.971840 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:49.971808    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"]
Jul 17 02:06:49.976372 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:49.976352    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:49.979028 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:49.979005    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"openshift-service-ca.crt\""
Jul 17 02:06:49.979856 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:49.979835    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"rate-test-model-1f2720-kserve-self-signed-certs\""
Jul 17 02:06:49.979960 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:49.979878    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"kube-root-ca.crt\""
Jul 17 02:06:49.979960 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:49.979937    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-1f2720\"/\"default-dockercfg-hpl7h\""
Jul 17 02:06:49.984441 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:49.984420    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"]
Jul 17 02:06:50.061920 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.061878    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tmp-dir\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.062105 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.061939    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwftk\" (UniqueName: \"kubernetes.io/projected/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-kube-api-access-fwftk\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.062105 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.062021    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-home\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.062105 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.062057    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tls-certs\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.062105 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.062083    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-dshm\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.062105 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.062105    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-model-cache\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.162787 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.162755    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tmp-dir\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.162969 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.162820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwftk\" (UniqueName: \"kubernetes.io/projected/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-kube-api-access-fwftk\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.162969 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.162866    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-home\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.162969 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.162897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tls-certs\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.162969 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.162925    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-dshm\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.163220 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.163002    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-model-cache\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.163314 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.163291    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tmp-dir\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.163463 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.163357    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-model-cache\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.163633 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.163613    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-home\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.165651 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.165621    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-dshm\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.165912 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.165893    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tls-certs\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.172890 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.172867    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwftk\" (UniqueName: \"kubernetes.io/projected/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-kube-api-access-fwftk\") pod \"rate-test-model-1f2720-kserve-cb749bddc-nd89m\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") " pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.286250 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.286153    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:06:50.420531 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.420483    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"]
Jul 17 02:06:50.422358 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:06:50.422330    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod170b9a29_fe48_4b7d_a4b6_4e30a1074b37.slice/crio-5c368846816574223dd3cae3be7dc30f0ec828a9800308e16d8f3875275274be WatchSource:0}: Error finding container 5c368846816574223dd3cae3be7dc30f0ec828a9800308e16d8f3875275274be: Status 404 returned error can't find the container with id 5c368846816574223dd3cae3be7dc30f0ec828a9800308e16d8f3875275274be
Jul 17 02:06:50.545287 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.545202    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m" event={"ID":"170b9a29-fe48-4b7d-a4b6-4e30a1074b37","Type":"ContainerStarted","Data":"08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092"}
Jul 17 02:06:50.545287 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.545253    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m" event={"ID":"170b9a29-fe48-4b7d-a4b6-4e30a1074b37","Type":"ContainerStarted","Data":"5c368846816574223dd3cae3be7dc30f0ec828a9800308e16d8f3875275274be"}
Jul 17 02:06:50.561589 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:50.561538    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m" podStartSLOduration=1.561517863 podStartE2EDuration="1.561517863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:06:50.560960208 +0000 UTC m=+3218.064646094" watchObservedRunningTime="2026-07-17 02:06:50.561517863 +0000 UTC m=+3218.065203727"
Jul 17 02:06:56.059048 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:56.059012    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:06:56.985635 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:56.985601    2582 scope.go:117] "RemoveContainer" containerID="bf44a587079c9c2585f60af4fbfec6c625eee9cfd0e8ea7aee9ab1cce03d14b7"
Jul 17 02:06:57.582021 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:57.581935    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" event={"ID":"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958","Type":"ContainerStarted","Data":"0b5699dfe4f4488da112459de8f517920f5a3386f961cc0100e6ff5eada41174"}
Jul 17 02:06:58.588689 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:58.588650    2582 generic.go:356] "Generic (PLEG): container finished" podID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerID="0b5699dfe4f4488da112459de8f517920f5a3386f961cc0100e6ff5eada41174" exitCode=6
Jul 17 02:06:58.589093 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:58.588717    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" event={"ID":"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958","Type":"ContainerDied","Data":"0b5699dfe4f4488da112459de8f517920f5a3386f961cc0100e6ff5eada41174"}
Jul 17 02:06:58.589093 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:58.588804    2582 scope.go:117] "RemoveContainer" containerID="bf44a587079c9c2585f60af4fbfec6c625eee9cfd0e8ea7aee9ab1cce03d14b7"
Jul 17 02:06:58.589093 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:58.589088    2582 scope.go:117] "RemoveContainer" containerID="0b5699dfe4f4488da112459de8f517920f5a3386f961cc0100e6ff5eada41174"
Jul 17 02:06:58.589350 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:06:58.589326    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw_odh-ai-gateway-infra(12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958"
Jul 17 02:06:58.613041 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:58.613017    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"]
Jul 17 02:06:59.736744 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:59.736721    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"
Jul 17 02:06:59.753054 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:59.753021    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958-maas-api-service-ca\") pod \"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958\" (UID: \"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958\") "
Jul 17 02:06:59.753393 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:59.753369    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" (UID: "12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 02:06:59.854321 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:06:59.854226    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:07:00.286996 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.286964    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:07:00.287253 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.287235    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:07:00.299804 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.299781    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:07:00.601439 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.601357    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"
Jul 17 02:07:00.601439 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.601405    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw" event={"ID":"12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958","Type":"ContainerDied","Data":"15ce8985ba14c5a59695422e572364983957b2842e7af05288d0d0fc3b4389e2"}
Jul 17 02:07:00.601695 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.601459    2582 scope.go:117] "RemoveContainer" containerID="0b5699dfe4f4488da112459de8f517920f5a3386f961cc0100e6ff5eada41174"
Jul 17 02:07:00.613896 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.613868    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:07:00.623243 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.623213    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"]
Jul 17 02:07:00.629039 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.629018    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-f0d41c05-ztknw"]
Jul 17 02:07:00.990546 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:00.990514    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" path="/var/lib/kubelet/pods/12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958/volumes"
Jul 17 02:07:10.411038 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:10.410958    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:07:26.358014 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:26.357981    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:07:34.751712 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:34.751679    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"]
Jul 17 02:07:41.657312 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:41.657273    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:07:49.772014 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:49.771970    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m" podUID="170b9a29-fe48-4b7d-a4b6-4e30a1074b37" containerName="main" containerID="cri-o://08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092" gracePeriod=45
Jul 17 02:07:50.038891 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.038867    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:07:50.187769 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.187736    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-home\") pod \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") "
Jul 17 02:07:50.187973 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.187794    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwftk\" (UniqueName: \"kubernetes.io/projected/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-kube-api-access-fwftk\") pod \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") "
Jul 17 02:07:50.187973 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.187837    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tls-certs\") pod \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") "
Jul 17 02:07:50.187973 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.187863    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-dshm\") pod \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") "
Jul 17 02:07:50.187973 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.187894    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tmp-dir\") pod \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") "
Jul 17 02:07:50.187973 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.187947    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-model-cache\") pod \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\" (UID: \"170b9a29-fe48-4b7d-a4b6-4e30a1074b37\") "
Jul 17 02:07:50.188243 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.188052    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-home" (OuterVolumeSpecName: "home") pod "170b9a29-fe48-4b7d-a4b6-4e30a1074b37" (UID: "170b9a29-fe48-4b7d-a4b6-4e30a1074b37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:07:50.188243 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.188201    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "170b9a29-fe48-4b7d-a4b6-4e30a1074b37" (UID: "170b9a29-fe48-4b7d-a4b6-4e30a1074b37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:07:50.188243 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.188211    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-model-cache" (OuterVolumeSpecName: "model-cache") pod "170b9a29-fe48-4b7d-a4b6-4e30a1074b37" (UID: "170b9a29-fe48-4b7d-a4b6-4e30a1074b37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:07:50.188391 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.188299    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tmp-dir\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:07:50.188391 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.188315    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-model-cache\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:07:50.188391 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.188324    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-home\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:07:50.190159 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.190136    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-dshm" (OuterVolumeSpecName: "dshm") pod "170b9a29-fe48-4b7d-a4b6-4e30a1074b37" (UID: "170b9a29-fe48-4b7d-a4b6-4e30a1074b37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:07:50.190652 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.190634    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "170b9a29-fe48-4b7d-a4b6-4e30a1074b37" (UID: "170b9a29-fe48-4b7d-a4b6-4e30a1074b37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 02:07:50.190652 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.190646    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-kube-api-access-fwftk" (OuterVolumeSpecName: "kube-api-access-fwftk") pod "170b9a29-fe48-4b7d-a4b6-4e30a1074b37" (UID: "170b9a29-fe48-4b7d-a4b6-4e30a1074b37"). InnerVolumeSpecName "kube-api-access-fwftk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:07:50.289049 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.288967    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwftk\" (UniqueName: \"kubernetes.io/projected/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-kube-api-access-fwftk\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:07:50.289049 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.288998    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-tls-certs\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:07:50.289049 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.289009    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/170b9a29-fe48-4b7d-a4b6-4e30a1074b37-dshm\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:07:50.834882 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.834850    2582 generic.go:356] "Generic (PLEG): container finished" podID="170b9a29-fe48-4b7d-a4b6-4e30a1074b37" containerID="08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092" exitCode=0
Jul 17 02:07:50.835344 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.834896    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m" event={"ID":"170b9a29-fe48-4b7d-a4b6-4e30a1074b37","Type":"ContainerDied","Data":"08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092"}
Jul 17 02:07:50.835344 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.834909    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"
Jul 17 02:07:50.835344 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.834923    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m" event={"ID":"170b9a29-fe48-4b7d-a4b6-4e30a1074b37","Type":"ContainerDied","Data":"5c368846816574223dd3cae3be7dc30f0ec828a9800308e16d8f3875275274be"}
Jul 17 02:07:50.835344 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.834938    2582 scope.go:117] "RemoveContainer" containerID="08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092"
Jul 17 02:07:50.843980 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.843963    2582 scope.go:117] "RemoveContainer" containerID="08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092"
Jul 17 02:07:50.844215 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:07:50.844196    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092\": container with ID starting with 08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092 not found: ID does not exist" containerID="08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092"
Jul 17 02:07:50.844254 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.844222    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092"} err="failed to get container status \"08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092\": rpc error: code = NotFound desc = could not find container \"08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092\": container with ID starting with 08ad9e34ecc0dadc86abd139ea57b5c7fab937dc21eb4c57ca7563d93eb16092 not found: ID does not exist"
Jul 17 02:07:50.855920 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.855887    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"]
Jul 17 02:07:50.859324 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.859301    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-1f2720/rate-test-model-1f2720-kserve-cb749bddc-nd89m"]
Jul 17 02:07:50.990094 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:07:50.990062    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="170b9a29-fe48-4b7d-a4b6-4e30a1074b37" path="/var/lib/kubelet/pods/170b9a29-fe48-4b7d-a4b6-4e30a1074b37/volumes"
Jul 17 02:08:07.059338 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:07.059301    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:08:14.448483 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:14.448453    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 02:08:14.455329 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:14.455306    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 02:08:14.456582 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:14.456563    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 02:08:14.462855 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:14.462839    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s6hjc_c8690b31-f48e-4eec-aafb-c91fb3cf442a/ovn-acl-logging/0.log"
Jul 17 02:08:33.168458 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:33.168423    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qmzhb"]
Jul 17 02:08:45.360442 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:45.360352    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"]
Jul 17 02:08:45.360957 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:45.360674    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy" containerID="cri-o://fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0" gracePeriod=30
Jul 17 02:08:50.615202 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.615177    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:08:50.714553 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714524    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-credential-socket\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.714717 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714608    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-certs\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.714717 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714633    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-envoy\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.714717 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714669    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-podinfo\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.714886 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714719    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-token\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.714886 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714741    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c27ac79b-331d-4675-9b9e-63c3f3095522-istiod-ca-cert\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.714886 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714776    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-socket\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.714886 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714808    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-data\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.714886 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714814    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:08:50.714886 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714835    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-225jr\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-kube-api-access-225jr\") pod \"c27ac79b-331d-4675-9b9e-63c3f3095522\" (UID: \"c27ac79b-331d-4675-9b9e-63c3f3095522\") "
Jul 17 02:08:50.715196 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.714926    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:08:50.715196 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.715133    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-credential-socket\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:50.715196 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.715154    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-certs\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:50.715196 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.715152    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c27ac79b-331d-4675-9b9e-63c3f3095522-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 02:08:50.715605 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.715332    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:08:50.715768 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.715748    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-data" (OuterVolumeSpecName: "istio-data") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:08:50.717180 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.717151    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-kube-api-access-225jr" (OuterVolumeSpecName: "kube-api-access-225jr") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "kube-api-access-225jr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:08:50.717286 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.717230    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-token" (OuterVolumeSpecName: "istio-token") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 02:08:50.717286 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.717239    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 02:08:50.717286 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.717247    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c27ac79b-331d-4675-9b9e-63c3f3095522" (UID: "c27ac79b-331d-4675-9b9e-63c3f3095522"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 02:08:50.815769 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.815737    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-envoy\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:50.815769 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.815763    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-podinfo\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:50.815769 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.815776    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-token\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:50.815992 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.815789    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c27ac79b-331d-4675-9b9e-63c3f3095522-istiod-ca-cert\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:50.815992 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.815797    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-workload-socket\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:50.815992 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.815805    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c27ac79b-331d-4675-9b9e-63c3f3095522-istio-data\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:50.815992 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:50.815826    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-225jr\" (UniqueName: \"kubernetes.io/projected/c27ac79b-331d-4675-9b9e-63c3f3095522-kube-api-access-225jr\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:08:51.110466 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.110430    2582 generic.go:356] "Generic (PLEG): container finished" podID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerID="fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0" exitCode=0
Jul 17 02:08:51.110671 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.110481    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"
Jul 17 02:08:51.110671 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.110522    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" event={"ID":"c27ac79b-331d-4675-9b9e-63c3f3095522","Type":"ContainerDied","Data":"fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0"}
Jul 17 02:08:51.110671 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.110581    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j" event={"ID":"c27ac79b-331d-4675-9b9e-63c3f3095522","Type":"ContainerDied","Data":"44cd164c34bd47a0c8b7dadfbb6bd35eba4b6845d803bf564d2f62a600f53ffd"}
Jul 17 02:08:51.110671 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.110606    2582 scope.go:117] "RemoveContainer" containerID="fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0"
Jul 17 02:08:51.119188 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.119169    2582 scope.go:117] "RemoveContainer" containerID="d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648"
Jul 17 02:08:51.126908 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.126887    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"]
Jul 17 02:08:51.127185 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.127170    2582 scope.go:117] "RemoveContainer" containerID="fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0"
Jul 17 02:08:51.127392 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:08:51.127373    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0\": container with ID starting with fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0 not found: ID does not exist" containerID="fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0"
Jul 17 02:08:51.127445 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.127405    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0"} err="failed to get container status \"fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0\": rpc error: code = NotFound desc = could not find container \"fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0\": container with ID starting with fdac116c957a50996deb204ff19d3ace6fca2ceb9dcf9fe9a63eaa4c21f42aa0 not found: ID does not exist"
Jul 17 02:08:51.127445 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.127426    2582 scope.go:117] "RemoveContainer" containerID="d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648"
Jul 17 02:08:51.127682 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:08:51.127668    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648\": container with ID starting with d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648 not found: ID does not exist" containerID="d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648"
Jul 17 02:08:51.127737 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.127685    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648"} err="failed to get container status \"d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648\": rpc error: code = NotFound desc = could not find container \"d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648\": container with ID starting with d1836e49c0b47a2e9528500d834e70f08b4e92bba1ca7e0a58815960dae19648 not found: ID does not exist"
Jul 17 02:08:51.134011 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.133991    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-1f2720-openshift-default-7c7cc6995-xc65j"]
Jul 17 02:08:51.457810 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.457778    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8fc9m_6989d7c1-045d-450d-b55e-f1c329c1c59c/manager/0.log"
Jul 17 02:08:51.584160 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:51.584127    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-chljj_951173fb-ea92-4a52-b7a5-b5453d9d16b0/manager/0.log"
Jul 17 02:08:52.067959 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:52.067926    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5_faf3187d-a6c7-4550-90c2-843b37a09f8f/copy-manifests/0.log"
Jul 17 02:08:52.087277 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:52.087239    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5_faf3187d-a6c7-4550-90c2-843b37a09f8f/manager/0.log"
Jul 17 02:08:52.991611 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:52.991581    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" path="/var/lib/kubelet/pods/c27ac79b-331d-4675-9b9e-63c3f3095522/volumes"
Jul 17 02:08:56.060906 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:56.060874    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qmzhb_338ade41-85c7-47f4-b972-0fa46b25ee98/limitador/0.log"
Jul 17 02:08:56.542245 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:56.542212    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mm6t5_eff98150-13ba-4bc0-a104-6ff7b4a177d3/istio-proxy/0.log"
Jul 17 02:08:57.390937 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:57.390909    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f5b59d7fb-t6h6c_59aea58b-5006-4136-a5e5-76b3cc757e1d/router/0.log"
Jul 17 02:08:57.844828 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:57.844793    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qnn7t_6956ac5e-0202-48c2-9f0c-e5be0e3014c4/main/0.log"
Jul 17 02:08:58.071090 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:08:58.071063    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db425tl_46087fb2-c035-48d5-a42a-da522bc1c7a5/main/0.log"
Jul 17 02:09:07.032656 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:07.032625    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-flbd5_b09b456f-062e-4b27-b5cc-4cfaff662554/global-pull-secret-syncer/0.log"
Jul 17 02:09:07.176750 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:07.176723    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jlch5_a5f2c00d-8aa2-4181-99df-f099066c8a5b/konnectivity-agent/0.log"
Jul 17 02:09:07.278708 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:07.278679    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-254.ec2.internal_7f3251c29c2be847090021e12c41abdd/haproxy/0.log"
Jul 17 02:09:10.646988 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.646954    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"]
Jul 17 02:09:10.647373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647311    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy"
Jul 17 02:09:10.647373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647323    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy"
Jul 17 02:09:10.647373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647332    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:10.647373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647337    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:10.647373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647353    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy"
Jul 17 02:09:10.647373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647358    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy"
Jul 17 02:09:10.647373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647369    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="170b9a29-fe48-4b7d-a4b6-4e30a1074b37" containerName="main"
Jul 17 02:09:10.647373 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647374    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="170b9a29-fe48-4b7d-a4b6-4e30a1074b37" containerName="main"
Jul 17 02:09:10.647661 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647387    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:10.647661 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647392    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:10.647661 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647469    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy"
Jul 17 02:09:10.647661 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647477    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c27ac79b-331d-4675-9b9e-63c3f3095522" containerName="istio-proxy"
Jul 17 02:09:10.647661 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647484    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:10.647661 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647503    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:10.647661 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.647508    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="170b9a29-fe48-4b7d-a4b6-4e30a1074b37" containerName="main"
Jul 17 02:09:10.651760 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.651729    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"
Jul 17 02:09:10.655282 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.655032    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qvpl\""
Jul 17 02:09:10.655941 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.655914    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 02:09:10.658396 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.658371    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"]
Jul 17 02:09:10.787219 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.787172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37e56c5-d76c-44c1-90f8-87c83821fac2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2\" (UID: \"a37e56c5-d76c-44c1-90f8-87c83821fac2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"
Jul 17 02:09:10.888171 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.888134    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37e56c5-d76c-44c1-90f8-87c83821fac2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2\" (UID: \"a37e56c5-d76c-44c1-90f8-87c83821fac2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"
Jul 17 02:09:10.888774 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.888757    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37e56c5-d76c-44c1-90f8-87c83821fac2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2\" (UID: \"a37e56c5-d76c-44c1-90f8-87c83821fac2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"
Jul 17 02:09:10.966334 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:10.966299    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"
Jul 17 02:09:11.097957 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:11.097928    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"]
Jul 17 02:09:11.101794 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:09:11.101766    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37e56c5_d76c_44c1_90f8_87c83821fac2.slice/crio-76c42e3da0808d10de151ba345b45c2b7d6e0848039c7e176886967609dfa813 WatchSource:0}: Error finding container 76c42e3da0808d10de151ba345b45c2b7d6e0848039c7e176886967609dfa813: Status 404 returned error can't find the container with id 76c42e3da0808d10de151ba345b45c2b7d6e0848039c7e176886967609dfa813
Jul 17 02:09:11.203674 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:11.203643    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" event={"ID":"a37e56c5-d76c-44c1-90f8-87c83821fac2","Type":"ContainerStarted","Data":"76c42e3da0808d10de151ba345b45c2b7d6e0848039c7e176886967609dfa813"}
Jul 17 02:09:12.209946 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:12.209911    2582 generic.go:356] "Generic (PLEG): container finished" podID="a37e56c5-d76c-44c1-90f8-87c83821fac2" containerID="ce9aff8ff140ba8513278a5132452d39fc3fadb7ffd6119090e06326097d4067" exitCode=6
Jul 17 02:09:12.210337 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:12.209979    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" event={"ID":"a37e56c5-d76c-44c1-90f8-87c83821fac2","Type":"ContainerDied","Data":"ce9aff8ff140ba8513278a5132452d39fc3fadb7ffd6119090e06326097d4067"}
Jul 17 02:09:12.210337 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:12.210180    2582 scope.go:117] "RemoveContainer" containerID="ce9aff8ff140ba8513278a5132452d39fc3fadb7ffd6119090e06326097d4067"
Jul 17 02:09:12.859426 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:12.859396    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qmzhb_338ade41-85c7-47f4-b972-0fa46b25ee98/limitador/0.log"
Jul 17 02:09:13.217527 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:13.217475    2582 generic.go:356] "Generic (PLEG): container finished" podID="a37e56c5-d76c-44c1-90f8-87c83821fac2" containerID="d919ee465ff558b3eee96586b81eba7817458a87e7720ce013c10ec5c5410f56" exitCode=6
Jul 17 02:09:13.218010 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:13.217533    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" event={"ID":"a37e56c5-d76c-44c1-90f8-87c83821fac2","Type":"ContainerDied","Data":"d919ee465ff558b3eee96586b81eba7817458a87e7720ce013c10ec5c5410f56"}
Jul 17 02:09:13.218010 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:13.217587    2582 scope.go:117] "RemoveContainer" containerID="ce9aff8ff140ba8513278a5132452d39fc3fadb7ffd6119090e06326097d4067"
Jul 17 02:09:13.218010 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:13.217789    2582 scope.go:117] "RemoveContainer" containerID="d919ee465ff558b3eee96586b81eba7817458a87e7720ce013c10ec5c5410f56"
Jul 17 02:09:13.218184 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:09:13.218043    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2_odh-ai-gateway-infra(a37e56c5-d76c-44c1-90f8-87c83821fac2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" podUID="a37e56c5-d76c-44c1-90f8-87c83821fac2"
Jul 17 02:09:14.227099 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:14.227073    2582 scope.go:117] "RemoveContainer" containerID="d919ee465ff558b3eee96586b81eba7817458a87e7720ce013c10ec5c5410f56"
Jul 17 02:09:14.227512 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:09:14.227284    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2_odh-ai-gateway-infra(a37e56c5-d76c-44c1-90f8-87c83821fac2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" podUID="a37e56c5-d76c-44c1-90f8-87c83821fac2"
Jul 17 02:09:14.766965 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:14.766934    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-ftjxh_7f62a271-55ea-42a7-8bcf-feaf6b7f80c2/cluster-monitoring-operator/0.log"
Jul 17 02:09:15.055073 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:15.054992    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xz72l_dac01c22-87b3-48bc-8bd9-53978f872b7b/node-exporter/0.log"
Jul 17 02:09:15.073531 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:15.073509    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xz72l_dac01c22-87b3-48bc-8bd9-53978f872b7b/kube-rbac-proxy/0.log"
Jul 17 02:09:15.090939 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:15.090910    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xz72l_dac01c22-87b3-48bc-8bd9-53978f872b7b/init-textfile/0.log"
Jul 17 02:09:15.460113 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:15.460081    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-cttwl_4f66694f-050f-4a99-a9d4-a4f26f4a2550/prometheus-operator-admission-webhook/0.log"
Jul 17 02:09:16.803207 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:16.803179    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-7pr76_e6dd53d6-002a-42bf-83bb-81cecb4c2e68/networking-console-plugin/0.log"
Jul 17 02:09:17.244025 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.243980    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"]
Jul 17 02:09:17.244389 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.244373    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:17.244389 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.244390    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:17.244546 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.244455    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="12b3b17e-4e79-4f0e-bfcb-fe6b1c5c7958" containerName="revoke-keys"
Jul 17 02:09:17.248661 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.248643    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.251137 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.251117    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l2j4k\"/\"kube-root-ca.crt\""
Jul 17 02:09:17.251224 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.251157    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-l2j4k\"/\"default-dockercfg-5srgd\""
Jul 17 02:09:17.251979 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.251964    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l2j4k\"/\"openshift-service-ca.crt\""
Jul 17 02:09:17.258179 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.258159    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"]
Jul 17 02:09:17.344262 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.344230    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-podres\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.344464 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.344279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-lib-modules\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.344464 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.344379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bqvn\" (UniqueName: \"kubernetes.io/projected/cc7d254d-59dc-41bc-af6e-20fc81788506-kube-api-access-7bqvn\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.344464 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.344431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-proc\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.344649 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.344533    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-sys\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445286 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-proc\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445467 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445298    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-sys\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445467 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445322    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-podres\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445467 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445348    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-lib-modules\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445467 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445391    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-sys\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445467 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445415    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bqvn\" (UniqueName: \"kubernetes.io/projected/cc7d254d-59dc-41bc-af6e-20fc81788506-kube-api-access-7bqvn\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445467 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445391    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-proc\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445467 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445456    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-podres\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.445756 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.445534    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc7d254d-59dc-41bc-af6e-20fc81788506-lib-modules\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.450083 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.450054    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/2.log"
Jul 17 02:09:17.453899 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.453877    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bqvn\" (UniqueName: \"kubernetes.io/projected/cc7d254d-59dc-41bc-af6e-20fc81788506-kube-api-access-7bqvn\") pod \"perf-node-gather-daemonset-8d6cd\" (UID: \"cc7d254d-59dc-41bc-af6e-20fc81788506\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.459378 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.459354    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-ghjrn_50afa276-24c6-4cfe-b4e0-4d995de91eea/console-operator/3.log"
Jul 17 02:09:17.558538 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.558405    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:17.698196 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:17.698164    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"]
Jul 17 02:09:17.700451 ip-10-0-134-254 kubenswrapper[2582]: W0717 02:09:17.700423    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcc7d254d_59dc_41bc_af6e_20fc81788506.slice/crio-f18b7a5b252125bc3e35ffa650704b497b6d73b3bd80655c8687d1d4cfcc4e2f WatchSource:0}: Error finding container f18b7a5b252125bc3e35ffa650704b497b6d73b3bd80655c8687d1d4cfcc4e2f: Status 404 returned error can't find the container with id f18b7a5b252125bc3e35ffa650704b497b6d73b3bd80655c8687d1d4cfcc4e2f
Jul 17 02:09:18.252580 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:18.252536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd" event={"ID":"cc7d254d-59dc-41bc-af6e-20fc81788506","Type":"ContainerStarted","Data":"fc243d6a41c2c0d5cbc8e702d24d79a3e7b5ee85cb324ab2085e03b6e727c973"}
Jul 17 02:09:18.253023 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:18.252604    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd" event={"ID":"cc7d254d-59dc-41bc-af6e-20fc81788506","Type":"ContainerStarted","Data":"f18b7a5b252125bc3e35ffa650704b497b6d73b3bd80655c8687d1d4cfcc4e2f"}
Jul 17 02:09:18.253023 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:18.252629    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:18.267404 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:18.267358    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd" podStartSLOduration=1.2673443070000001 podStartE2EDuration="1.267344307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 02:09:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:09:18.266738863 +0000 UTC m=+3365.770424716" watchObservedRunningTime="2026-07-17 02:09:18.267344307 +0000 UTC m=+3365.771030168"
Jul 17 02:09:18.657355 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:18.657266    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-p6wz7_08b28450-98da-41fc-b59b-9ff001cb54c3/volume-data-source-validator/0.log"
Jul 17 02:09:19.654303 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:19.654272    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ks5wl_f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370/dns/0.log"
Jul 17 02:09:19.673143 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:19.673118    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ks5wl_f6bb9c9a-3b46-4a2f-9ec3-ee7fda9ab370/kube-rbac-proxy/0.log"
Jul 17 02:09:19.805588 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:19.805564    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8g97m_e600bd66-c18f-4b53-8f5d-dfff57448245/dns-node-resolver/0.log"
Jul 17 02:09:20.364207 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:20.364173    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5bmsd_ff931d44-0f32-4884-87a5-0d657a1afda9/node-ca/0.log"
Jul 17 02:09:21.355737 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:21.355705    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mm6t5_eff98150-13ba-4bc0-a104-6ff7b4a177d3/istio-proxy/0.log"
Jul 17 02:09:21.807050 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:21.807022    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f5b59d7fb-t6h6c_59aea58b-5006-4136-a5e5-76b3cc757e1d/router/0.log"
Jul 17 02:09:22.541738 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:22.541710    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7m99b_6332dd34-b9db-4816-ac4b-04e7a4de05ca/serve-healthcheck-canary/0.log"
Jul 17 02:09:23.048092 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:23.048062    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-l9lkq_32a81af9-c586-4ec0-b2a1-dca8d44df4fd/insights-operator/0.log"
Jul 17 02:09:23.049654 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:23.049634    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-l9lkq_32a81af9-c586-4ec0-b2a1-dca8d44df4fd/insights-operator/1.log"
Jul 17 02:09:23.216548 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:23.216445    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xtx4c_6573de58-3849-4424-a40d-52deb60da3cb/kube-rbac-proxy/0.log"
Jul 17 02:09:23.237239 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:23.237217    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xtx4c_6573de58-3849-4424-a40d-52deb60da3cb/exporter/0.log"
Jul 17 02:09:23.259596 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:23.259572    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xtx4c_6573de58-3849-4424-a40d-52deb60da3cb/extractor/0.log"
Jul 17 02:09:24.266940 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:24.266913    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-8d6cd"
Jul 17 02:09:25.354986 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:25.354951    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8fc9m_6989d7c1-045d-450d-b55e-f1c329c1c59c/manager/0.log"
Jul 17 02:09:25.610639 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:25.610552    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-chljj_951173fb-ea92-4a52-b7a5-b5453d9d16b0/manager/0.log"
Jul 17 02:09:25.984592 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:25.984562    2582 scope.go:117] "RemoveContainer" containerID="d919ee465ff558b3eee96586b81eba7817458a87e7720ce013c10ec5c5410f56"
Jul 17 02:09:26.072288 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:26.072250    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5_faf3187d-a6c7-4550-90c2-843b37a09f8f/manager/0.log"
Jul 17 02:09:26.088293 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:26.088265    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56ddbf9bf4-sg7c5_faf3187d-a6c7-4550-90c2-843b37a09f8f/copy-manifests/0.log"
Jul 17 02:09:26.290303 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:26.290211    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" event={"ID":"a37e56c5-d76c-44c1-90f8-87c83821fac2","Type":"ContainerStarted","Data":"9335cab3b438449ef5c432aa8e5c96ed1ad8b1799625b03e629da96fa0ad3a89"}
Jul 17 02:09:26.305287 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:26.305231    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" podStartSLOduration=16.305216578 podStartE2EDuration="16.305216578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:09:26.304268102 +0000 UTC m=+3373.807953964" watchObservedRunningTime="2026-07-17 02:09:26.305216578 +0000 UTC m=+3373.808902440"
Jul 17 02:09:27.296518 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:27.296460    2582 generic.go:356] "Generic (PLEG): container finished" podID="a37e56c5-d76c-44c1-90f8-87c83821fac2" containerID="9335cab3b438449ef5c432aa8e5c96ed1ad8b1799625b03e629da96fa0ad3a89" exitCode=6
Jul 17 02:09:27.297012 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:27.296542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" event={"ID":"a37e56c5-d76c-44c1-90f8-87c83821fac2","Type":"ContainerDied","Data":"9335cab3b438449ef5c432aa8e5c96ed1ad8b1799625b03e629da96fa0ad3a89"}
Jul 17 02:09:27.297012 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:27.296603    2582 scope.go:117] "RemoveContainer" containerID="d919ee465ff558b3eee96586b81eba7817458a87e7720ce013c10ec5c5410f56"
Jul 17 02:09:27.297012 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:27.296889    2582 scope.go:117] "RemoveContainer" containerID="9335cab3b438449ef5c432aa8e5c96ed1ad8b1799625b03e629da96fa0ad3a89"
Jul 17 02:09:27.297169 ip-10-0-134-254 kubenswrapper[2582]: E0717 02:09:27.297137    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2_odh-ai-gateway-infra(a37e56c5-d76c-44c1-90f8-87c83821fac2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" podUID="a37e56c5-d76c-44c1-90f8-87c83821fac2"
Jul 17 02:09:27.319551 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:27.319527    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"]
Jul 17 02:09:27.563145 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:27.563054    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66ffdbf46d-gn45x_deff0458-dd21-45ee-937a-f0b830492acc/manager/0.log"
Jul 17 02:09:27.847294 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:27.847183    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mm6t5_eff98150-13ba-4bc0-a104-6ff7b4a177d3/istio-proxy/0.log"
Jul 17 02:09:28.236806 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:28.236761    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f5b59d7fb-t6h6c_59aea58b-5006-4136-a5e5-76b3cc757e1d/router/0.log"
Jul 17 02:09:28.443150 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:28.443129    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"
Jul 17 02:09:28.537869 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:28.537782    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37e56c5-d76c-44c1-90f8-87c83821fac2-maas-api-service-ca\") pod \"a37e56c5-d76c-44c1-90f8-87c83821fac2\" (UID: \"a37e56c5-d76c-44c1-90f8-87c83821fac2\") "
Jul 17 02:09:28.538120 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:28.538097    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a37e56c5-d76c-44c1-90f8-87c83821fac2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a37e56c5-d76c-44c1-90f8-87c83821fac2" (UID: "a37e56c5-d76c-44c1-90f8-87c83821fac2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 02:09:28.639217 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:28.639185    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a37e56c5-d76c-44c1-90f8-87c83821fac2-maas-api-service-ca\") on node \"ip-10-0-134-254.ec2.internal\" DevicePath \"\""
Jul 17 02:09:29.309992 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:29.309894    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2" event={"ID":"a37e56c5-d76c-44c1-90f8-87c83821fac2","Type":"ContainerDied","Data":"76c42e3da0808d10de151ba345b45c2b7d6e0848039c7e176886967609dfa813"}
Jul 17 02:09:29.309992 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:29.309962    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"
Jul 17 02:09:29.309992 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:29.309974    2582 scope.go:117] "RemoveContainer" containerID="9335cab3b438449ef5c432aa8e5c96ed1ad8b1799625b03e629da96fa0ad3a89"
Jul 17 02:09:29.325875 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:29.325853    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"]
Jul 17 02:09:29.333204 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:29.333184    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d7122b7f-tlbv2"]
Jul 17 02:09:29.403671 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:29.403639    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mm6t5_eff98150-13ba-4bc0-a104-6ff7b4a177d3/istio-proxy/0.log"
Jul 17 02:09:29.792455 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:29.792427    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5f5b59d7fb-t6h6c_59aea58b-5006-4136-a5e5-76b3cc757e1d/router/0.log"
Jul 17 02:09:30.990932 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:30.990888    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a37e56c5-d76c-44c1-90f8-87c83821fac2" path="/var/lib/kubelet/pods/a37e56c5-d76c-44c1-90f8-87c83821fac2/volumes"
Jul 17 02:09:32.557636 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:32.557604    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-rbqqj_e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad/migrator/0.log"
Jul 17 02:09:32.574707 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:32.574677    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-rbqqj_e8cc3b3c-99d5-482a-8ef9-c468ca5cb3ad/graceful-termination/0.log"
Jul 17 02:09:32.908351 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:32.908265    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-9xsvj_5b704339-44a2-4e1c-96b1-3950ca6a5d13/kube-storage-version-migrator-operator/1.log"
Jul 17 02:09:32.910137 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:32.910101    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-9xsvj_5b704339-44a2-4e1c-96b1-3950ca6a5d13/kube-storage-version-migrator-operator/0.log"
Jul 17 02:09:34.049903 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:34.049877    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxbh2_c75f5bce-b93f-4a73-8119-63546da6f555/kube-multus-additional-cni-plugins/0.log"
Jul 17 02:09:34.069915 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:34.069882    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxbh2_c75f5bce-b93f-4a73-8119-63546da6f555/egress-router-binary-copy/0.log"
Jul 17 02:09:34.089755 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:34.089729    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxbh2_c75f5bce-b93f-4a73-8119-63546da6f555/cni-plugins/0.log"
Jul 17 02:09:34.112740 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:34.112713    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxbh2_c75f5bce-b93f-4a73-8119-63546da6f555/bond-cni-plugin/0.log"
Jul 17 02:09:34.139566 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:34.139530    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxbh2_c75f5bce-b93f-4a73-8119-63546da6f555/routeoverride-cni/0.log"
Jul 17 02:09:34.158214 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:34.158182    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxbh2_c75f5bce-b93f-4a73-8119-63546da6f555/whereabouts-cni-bincopy/0.log"
Jul 17 02:09:34.180077 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:34.180054    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pxbh2_c75f5bce-b93f-4a73-8119-63546da6f555/whereabouts-cni/0.log"
Jul 17 02:09:34.389354 ip-10-0-134-254 kubenswrapper[2582]: I0717 02:09:34.389274    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-blhf9_db00c4f8-4c3f-4788-85a3-8fbd47cc557d/kube-multus/0.log"