Jul 21 20:39:32.660289 ip-10-0-130-177 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 20:39:33.102833 ip-10-0-130-177 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 20:39:33.102833 ip-10-0-130-177 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 20:39:33.102833 ip-10-0-130-177 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 20:39:33.102833 ip-10-0-130-177 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 20:39:33.102833 ip-10-0-130-177 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 20:39:33.104931 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.104843    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 20:39:33.111355 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111338    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 20:39:33.111355 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111354    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111359    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111363    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111365    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111368    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111371    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111374    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111377    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111379    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111382    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111385    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111393    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111396    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111399    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111401    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111404    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111407    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111410    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111413    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111415    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 20:39:33.111434 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111418    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111420    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111423    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111426    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111428    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111431    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111434    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111437    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111440    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111443    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111447    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111451    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111454    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111457    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111460    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111463    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111465    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111468    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111471    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111474    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 20:39:33.111894 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111476    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111479    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111481    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111484    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111486    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111489    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111491    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111493    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111496    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111498    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111501    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111503    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111505    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111508    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111511    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111514    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111517    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111520    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111522    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111525    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 20:39:33.112386 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111527    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111530    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111532    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111535    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111537    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111540    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111542    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111545    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111548    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111551    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111554    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111557    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111559    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111561    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111564    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111566    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111569    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111571    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111574    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111576    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 20:39:33.112872 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111579    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111581    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111584    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111586    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111589    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111592    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111596    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111598    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111601    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111603    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111606    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111610    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111613    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111616    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111620    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111623    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111626    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.111629    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112349    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 20:39:33.113352 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112356    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112359    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112362    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112364    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112367    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112370    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112373    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112375    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112378    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112380    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112383    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112385    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112389    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112393    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112397    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112401    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112404    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112407    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112410    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112413    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 20:39:33.113793 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112416    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112419    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112422    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112424    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112427    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112429    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112432    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112435    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112437    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112440    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112443    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112445    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112448    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112450    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112453    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112455    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112457    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112460    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112465    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112468    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 20:39:33.114303 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112471    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112474    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112477    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112481    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112484    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112486    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112489    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112492    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112494    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112497    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112502    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112505    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112508    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112511    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112514    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112517    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112520    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112530    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112535    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 20:39:33.114791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112538    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112541    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112544    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112546    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112549    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112551    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112554    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112557    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112559    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112562    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112565    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112569    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112572    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112575    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112578    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112581    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112583    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112586    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112588    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112591    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 20:39:33.115270 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112594    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112596    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112599    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112601    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112604    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112607    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112610    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112612    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112615    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112617    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112620    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112622    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112625    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112627    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112630    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112633    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112635    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112638    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.112641    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113079    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 20:39:33.115745 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113086    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113090    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113093    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113096    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113099    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113102    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113105    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113107    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113110    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113113    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113115    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113118    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113121    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113123    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113125    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113128    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113132    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113136    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113139    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 20:39:33.116266 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113141    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113145    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113149    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113152    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113155    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113157    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113160    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113162    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113165    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113167    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113170    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113172    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113174    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113177    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113179    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113182    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113184    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113187    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113190    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 20:39:33.116711 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113192    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113195    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113197    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113199    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113202    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113204    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113207    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113209    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113212    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113214    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113217    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113220    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113223    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113225    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113227    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113231    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113233    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113236    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113239    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113241    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 20:39:33.117185 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113244    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113247    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113250    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113252    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113254    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113257    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113259    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113262    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113264    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113267    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113269    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113272    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113275    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113277    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113280    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113282    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113284    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113287    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113290    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113292    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 20:39:33.117692 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113295    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113297    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113300    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113303    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113306    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113308    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113311    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113313    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113316    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113319    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113322    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113324    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113327    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113329    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113331    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113334    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113338    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113341    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113344    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.113347    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 20:39:33.118176 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113416    2567 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113423    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113429    2567 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113434    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113438    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113441    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113445    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113449    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113453    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113455    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113462    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113465    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113468    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113471    2567 flags.go:64] FLAG: --cgroup-root=""
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113474    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113477    2567 flags.go:64] FLAG: --client-ca-file=""
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113480    2567 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113483    2567 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113487    2567 flags.go:64] FLAG: --cluster-domain=""
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113490    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113493    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113496    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113500    2567 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113503    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 20:39:33.118687 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113506    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113510    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113513    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113516    2567 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113519    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113522    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113525    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113528    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113532    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113535    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113538    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113541    2567 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113544    2567 flags.go:64] FLAG: --enable-server="true"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113547    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113551    2567 flags.go:64] FLAG: --event-burst="100"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113555    2567 flags.go:64] FLAG: --event-qps="50"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113558    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113560    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113565    2567 flags.go:64] FLAG: --eviction-hard=""
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113568    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113571    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113574    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113577    2567 flags.go:64] FLAG: --eviction-soft=""
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113580    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113583    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 20:39:33.119315 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113586    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113589    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113593    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113596    2567 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113599    2567 flags.go:64] FLAG: --feature-gates=""
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113602    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113605    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113608    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113611    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113614    2567 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113617    2567 flags.go:64] FLAG: --help="false"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113620    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113624    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113626    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113629    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113633    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113636    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113639    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113642    2567 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113645    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113647    2567 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113651    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113654    2567 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113657    2567 flags.go:64] FLAG: --kube-reserved=""
Jul 21 20:39:33.119929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113660    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113664    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113667    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113669    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113672    2567 flags.go:64] FLAG: --lock-file=""
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113675    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113678    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113681    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113686    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113689    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113692    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113695    2567 flags.go:64] FLAG: --logging-format="text"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113698    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113701    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113704    2567 flags.go:64] FLAG: --manifest-url=""
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113707    2567 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113711    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113714    2567 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113718    2567 flags.go:64] FLAG: --max-pods="110"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113721    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113724    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113727    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113729    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113732    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113735    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 20:39:33.120516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113738    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113745    2567 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113748    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113751    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113754    2567 flags.go:64] FLAG: --pod-cidr=""
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113757    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113763    2567 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113766    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113769    2567 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113773    2567 flags.go:64] FLAG: --port="10250"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113776    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113779    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0212314b049c1560b"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113782    2567 flags.go:64] FLAG: --qos-reserved=""
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113785    2567 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113788    2567 flags.go:64] FLAG: --register-node="true"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113791    2567 flags.go:64] FLAG: --register-with-taints=""
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113795    2567 flags.go:64] FLAG: --registry-burst="10"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113799    2567 flags.go:64] FLAG: --registry-qps="5"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113802    2567 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113805    2567 flags.go:64] FLAG: --reserved-memory=""
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113809    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113812    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113816    2567 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113818    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113821    2567 flags.go:64] FLAG: --runonce="false"
Jul 21 20:39:33.121213 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113824    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113827    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113830    2567 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113833    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113836    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113839    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113842    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113845    2567 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113847    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113850    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113853    2567 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113856    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113859    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113862    2567 flags.go:64] FLAG: --system-cgroups=""
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113865    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113870    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113874    2567 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113877    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113881    2567 flags.go:64] FLAG: --tls-min-version=""
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113884    2567 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113887    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113890    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113893    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113896    2567 flags.go:64] FLAG: --v="2"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113901    2567 flags.go:64] FLAG: --version="false"
Jul 21 20:39:33.121805 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113905    2567 flags.go:64] FLAG: --vmodule=""
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113908    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.113911    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114044    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114049    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114053    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114056    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114059    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114062    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114065    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114068    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114071    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114074    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114076    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114079    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114082    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114084    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114087    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114089    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114092    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 20:39:33.122424 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114094    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114097    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114100    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114105    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114108    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114111    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114113    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114116    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114118    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114121    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114123    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114127    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114130    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114132    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114135    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114137    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114141    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114144    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114148    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 20:39:33.122941 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114151    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114155    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114159    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114162    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114165    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114168    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114171    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114174    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114176    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114179    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114181    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114184    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114186    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114189    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114191    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114194    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114198    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114200    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114203    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114206    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 20:39:33.123413 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114208    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114210    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114213    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114215    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114220    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114222    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114225    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114227    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114230    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114232    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114235    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114237    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114240    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114242    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114244    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114247    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114250    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114252    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114255    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114257    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 20:39:33.123877 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114260    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114264    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114267    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114270    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114272    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114275    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114278    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114280    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114284    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114287    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114290    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114292    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114295    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114297    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114299    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114302    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114306    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114309    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114311    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 20:39:33.124377 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114314    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114317    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114319    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.114322    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.115232    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.122504    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.122521    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.122541    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.122546    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122609    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122614    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122618    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122622    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122625    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 20:39:33.124839 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122628    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122631    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122634    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122637    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122639    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122642    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122644    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122647    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122664    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122668    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122671    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122674    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122676    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122679    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122681    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122684    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122687    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122689    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122692    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122695    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 20:39:33.125202 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122697    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122700    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122702    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122705    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122713    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122716    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122718    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122721    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122723    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122726    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122728    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122731    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122733    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122735    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122738    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122740    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122742    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122745    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122747    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122750    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 20:39:33.125671 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122753    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122755    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122759    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122763    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122766    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122768    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122771    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122775    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122778    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122781    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122783    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122786    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122788    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122791    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122793    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122796    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122798    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122806    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122809    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 20:39:33.126231 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122812    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122814    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122817    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122819    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122822    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122824    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122827    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122829    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122832    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122834    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122837    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122839    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122842    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122845    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122847    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122850    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122852    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122855    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122857    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122860    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 20:39:33.126691 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122862    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122865    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122867    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122870    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122872    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122875    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122878    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122880    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122883    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122885    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122888    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122896    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122898    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122901    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.122904    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 20:39:33.127187 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.122908    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123075    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123081    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123084    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123087    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123090    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123093    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123096    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123098    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123101    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123104    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123107    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123109    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123112    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123115    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123117    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123120    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123122    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123125    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123127    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 20:39:33.127554 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123131    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123135    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123138    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123141    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123143    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123146    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123149    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123151    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123154    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123168    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123171    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123174    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123177    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123179    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123182    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123184    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123187    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123189    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123193    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 20:39:33.128133 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123196    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123199    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123202    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123205    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123208    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123210    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123213    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123215    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123217    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123220    2567 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123222    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123225    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123227    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123230    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123232    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123234    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123237    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123240    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123242    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123244    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 20:39:33.128612 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123247    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123249    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123252    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123261    2567 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123264    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123267    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123269    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123272    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123274    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123277    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123279    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123281    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123284    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123286    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123289    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123291    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123294    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123296    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123299    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123301    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 20:39:33.129104 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123304    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123306    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123309    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123311    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123314    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123316    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123318    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123321    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123323    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123326    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123328    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123331    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123333    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123335    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123338    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123340    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123349    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123351    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123354    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123357    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 20:39:33.129587 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:33.123361    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 20:39:33.130065 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.123365    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 20:39:33.130065 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.123488    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 20:39:33.130065 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.126011    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 20:39:33.130065 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.127736    2567 server.go:1019] "Starting client certificate rotation"
Jul 21 20:39:33.130065 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.127835    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 20:39:33.130065 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.128736    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 20:39:33.130655 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.130642    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 20:39:33.136186 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.136163    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 20:39:33.149268 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.149251    2567 log.go:25] "Validated CRI v1 runtime API"
Jul 21 20:39:33.151707 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.151693    2567 log.go:25] "Validated CRI v1 image API"
Jul 21 20:39:33.155042 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.155025    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 20:39:33.157519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.157502    2567 fs.go:135] Filesystem UUIDs: map[1e6c28a6-09ca-48f1-b0f9-7a57599e27c3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b34c8772-ea58-469f-9d63-538c5ef2880f:/dev/nvme0n1p3]
Jul 21 20:39:33.157570 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.157518    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 20:39:33.161285 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.161268    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 20:39:33.162525 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.162400    2567 manager.go:216] Machine: {Timestamp:2026-07-21 20:39:33.161900594 +0000 UTC m=+0.371929005 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104111 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a7da899912adcaaa6fa4a01e2d687 SystemUUID:ec2a7da8-9991-2adc-aaa6-fa4a01e2d687 BootID:ba6c8e40-8013-43d6-9945-17fd517a83d9 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d4:4b:29:31:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d4:4b:29:31:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:89:49:0c:b9:d6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 20:39:33.162525 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.162521    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 20:39:33.162629 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.162592    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 20:39:33.166057 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.166031    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 20:39:33.166188 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.166060    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-177.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 20:39:33.167558 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.167548    2567 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 20:39:33.167600 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.167560    2567 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 20:39:33.167600 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.167568    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 20:39:33.168253 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.168232    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 20:39:33.170192 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.170181    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 20:39:33.174935 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.174923    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 20:39:33.175061 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.175052    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 20:39:33.175100 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.175082    2567 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 20:39:33.175100 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.175092    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 20:39:33.175156 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.175102    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 20:39:33.175156 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.175113    2567 kubelet.go:398] "Adding apiserver pod source"
Jul 21 20:39:33.175156 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.175122    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 20:39:33.175529 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.175517    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 20:39:33.175575 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.175534    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 20:39:33.177781 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.177756    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 20:39:33.180371 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.180353    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 20:39:33.180455 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.180392    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 20:39:33.181744 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181730    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 20:39:33.181811 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181752    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 20:39:33.181811 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181762    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 20:39:33.181811 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181771    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 20:39:33.181811 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181784    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 20:39:33.181811 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181792    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 20:39:33.181811 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181804    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 20:39:33.181811 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181814    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 20:39:33.182057 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181825    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 20:39:33.182057 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181834    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 20:39:33.182057 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181847    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 20:39:33.182057 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.181857    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 20:39:33.182780 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.182769    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 20:39:33.182832 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.182783    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 20:39:33.186312 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.186285    2567 server.go:1268] "Started kubelet"
Jul 21 20:39:33.186561 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.186530    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 20:39:33.186635 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.186587    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 20:39:33.186723 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.186696    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 20:39:33.187155 ip-10-0-130-177 systemd[1]: Started Kubernetes Kubelet.
Jul 21 20:39:33.187877 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.187223    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 20:39:33.187877 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.187295    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-177.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 20:39:33.187877 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.187589    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-177.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 20:39:33.188718 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.188702    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 20:39:33.189919 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.189904    2567 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 20:39:33.192582 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.192551    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 20:39:33.196720 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.195637    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-177.ec2.internal.18c4689c6059f1ba  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-177.ec2.internal,UID:ip-10-0-130-177.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-177.ec2.internal,},FirstTimestamp:2026-07-21 20:39:33.186253242 +0000 UTC m=+0.396281653,LastTimestamp:2026-07-21 20:39:33.186253242 +0000 UTC m=+0.396281653,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-177.ec2.internal,}"
Jul 21 20:39:33.199034 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.199020    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 20:39:33.199134 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.199043    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 20:39:33.199180 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.199132    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:33.199862 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.199844    2567 factory.go:55] Registering systemd factory
Jul 21 20:39:33.199862 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.199864    2567 factory.go:223] Registration of the systemd container factory successfully
Jul 21 20:39:33.199980 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.199848    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 20:39:33.199980 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.199848    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 20:39:33.199980 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.199903    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 20:39:33.200152 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.200012    2567 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 20:39:33.200152 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.200022    2567 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 20:39:33.200240 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.200165    2567 factory.go:153] Registering CRI-O factory
Jul 21 20:39:33.200240 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.200176    2567 factory.go:223] Registration of the crio container factory successfully
Jul 21 20:39:33.200326 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.200240    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 20:39:33.200326 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.200260    2567 factory.go:103] Registering Raw factory
Jul 21 20:39:33.200326 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.200271    2567 manager.go:1196] Started watching for new ooms in manager
Jul 21 20:39:33.200585 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.200571    2567 manager.go:349] Starting recovery of all containers
Jul 21 20:39:33.208736 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.208712    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-177.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 20:39:33.208819 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.208791    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 20:39:33.210486 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.210470    2567 manager.go:354] Recovery completed
Jul 21 20:39:33.215115 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.215103    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 20:39:33.215589 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.215571    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dcm52"
Jul 21 20:39:33.216730 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.216714    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 20:39:33.216801 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.216743    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 20:39:33.216801 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.216754    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientPID"
Jul 21 20:39:33.217370 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.217357    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 20:39:33.217452 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.217367    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 20:39:33.217452 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.217393    2567 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 20:39:33.217452 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.217400    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 20:39:33.218200 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.218185    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 20:39:33.218263 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.218211    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 20:39:33.218263 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.218221    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientPID"
Jul 21 20:39:33.218580 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.218499    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-177.ec2.internal.18c4689c622afbb6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-177.ec2.internal,UID:ip-10-0-130-177.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-177.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-177.ec2.internal,},FirstTimestamp:2026-07-21 20:39:33.216730038 +0000 UTC m=+0.426758449,LastTimestamp:2026-07-21 20:39:33.216730038 +0000 UTC m=+0.426758449,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-177.ec2.internal,}"
Jul 21 20:39:33.219631 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.219617    2567 policy_none.go:49] "None policy: Start"
Jul 21 20:39:33.219690 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.219635    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 20:39:33.219690 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.219647    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 20:39:33.221662 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.221651    2567 policy_none.go:47] "Start"
Jul 21 20:39:33.224285 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.224136    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dcm52"
Jul 21 20:39:33.228915 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.228857    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-177.ec2.internal.18c4689c622b42ba  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-177.ec2.internal,UID:ip-10-0-130-177.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-177.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-177.ec2.internal,},FirstTimestamp:2026-07-21 20:39:33.216748218 +0000 UTC m=+0.426776630,LastTimestamp:2026-07-21 20:39:33.216748218 +0000 UTC m=+0.426776630,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-177.ec2.internal,}"
Jul 21 20:39:33.257643 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.257627    2567 manager.go:337] "Starting Device Plugin manager"
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.257661    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.257674    2567 server.go:87] "Starting device plugin registration server"
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.257952    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.257964    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.258068    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.258143    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.258151    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.258649    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 20:39:33.278621 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.258706    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:33.303495 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.303471    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 20:39:33.304585 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.304571    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 20:39:33.304665 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.304613    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 20:39:33.304665 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.304637    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 20:39:33.304762 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.304676    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 20:39:33.307142 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.307121    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 20:39:33.359072 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.359006    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 20:39:33.360029 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.360014    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 20:39:33.360108 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.360042    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 20:39:33.360108 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.360053    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientPID"
Jul 21 20:39:33.360108 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.360077    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.369054 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.369035    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.369165 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.369060    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-177.ec2.internal\": node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:33.397955 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.397931    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:33.405202 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.405173    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal"]
Jul 21 20:39:33.405284 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.405246    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 20:39:33.406461 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.406447    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 20:39:33.406529 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.406473    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 20:39:33.406529 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.406482    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientPID"
Jul 21 20:39:33.408876 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.408864    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 20:39:33.409027 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.409011    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.409073 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.409055    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 20:39:33.410000 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.409966    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 20:39:33.410092 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.410018    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 20:39:33.410092 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.410033    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientPID"
Jul 21 20:39:33.411214 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.411195    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 20:39:33.411297 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.411223    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 20:39:33.411297 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.411237    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientPID"
Jul 21 20:39:33.413586 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.413572    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.413655 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.413598    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 20:39:33.414475 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.414458    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 20:39:33.414574 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.414488    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 20:39:33.414574 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.414502    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientPID"
Jul 21 20:39:33.433567 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.433550    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-177.ec2.internal\" not found" node="ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.437865 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.437852    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-177.ec2.internal\" not found" node="ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.498428 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.498405    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:33.501681 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.501663    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e9f1b496a2d166fa089beef280b25f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal\" (UID: \"4e9f1b496a2d166fa089beef280b25f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.501740 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.501690    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e9f1b496a2d166fa089beef280b25f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal\" (UID: \"4e9f1b496a2d166fa089beef280b25f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.501740 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.501706    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1399074c7686ed18dafa282fd37a44d0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-177.ec2.internal\" (UID: \"1399074c7686ed18dafa282fd37a44d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.598871 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.598830    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:33.602128 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.602104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e9f1b496a2d166fa089beef280b25f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal\" (UID: \"4e9f1b496a2d166fa089beef280b25f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.602239 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.602135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e9f1b496a2d166fa089beef280b25f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal\" (UID: \"4e9f1b496a2d166fa089beef280b25f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.602239 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.602151    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1399074c7686ed18dafa282fd37a44d0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-177.ec2.internal\" (UID: \"1399074c7686ed18dafa282fd37a44d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.602239 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.602190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1399074c7686ed18dafa282fd37a44d0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-177.ec2.internal\" (UID: \"1399074c7686ed18dafa282fd37a44d0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.602239 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.602194    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e9f1b496a2d166fa089beef280b25f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal\" (UID: \"4e9f1b496a2d166fa089beef280b25f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.602239 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.602205    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e9f1b496a2d166fa089beef280b25f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal\" (UID: \"4e9f1b496a2d166fa089beef280b25f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.699536 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.699497    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:33.736894 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.736874    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.740273 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:33.740250    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:33.800641 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.800601    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:33.901029 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:33.901005    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:34.001444 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:34.001378    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:34.101765 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:34.101739    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:34.127034 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.127010    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 20:39:34.127391 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.127127    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 20:39:34.202449 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:34.202424    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:34.231195 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.231134    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 20:34:32 +0000 UTC" deadline="2028-04-25 13:05:54.069086842 +0000 UTC"
Jul 21 20:39:34.231195 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.231194    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15448h26m19.837897736s"
Jul 21 20:39:34.303565 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:34.303505    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:34.327457 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.327432    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 20:39:34.328542 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.328528    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 20:39:34.328629 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.328555    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 20:39:34.328629 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.328565    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeHasSufficientPID"
Jul 21 20:39:34.329102 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:34.329079    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1399074c7686ed18dafa282fd37a44d0.slice/crio-b334cf30934af329601a945e20fdd4f344824e97b756bf004cb45bb2bdf1bf5e WatchSource:0}: Error finding container b334cf30934af329601a945e20fdd4f344824e97b756bf004cb45bb2bdf1bf5e: Status 404 returned error can't find the container with id b334cf30934af329601a945e20fdd4f344824e97b756bf004cb45bb2bdf1bf5e
Jul 21 20:39:34.329403 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:34.329387    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9f1b496a2d166fa089beef280b25f7.slice/crio-7868b9f8d27d87b3085add03b0e6385c4116644129c336865f888f00e9701df6 WatchSource:0}: Error finding container 7868b9f8d27d87b3085add03b0e6385c4116644129c336865f888f00e9701df6: Status 404 returned error can't find the container with id 7868b9f8d27d87b3085add03b0e6385c4116644129c336865f888f00e9701df6
Jul 21 20:39:34.335092 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.335080    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 20:39:34.404500 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:34.404472    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:34.504930 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:34.504901    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-177.ec2.internal\" not found"
Jul 21 20:39:34.537663 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.537645    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 20:39:34.564663 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.564491    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 20:39:34.599147 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.599120    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:34.626682 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.626656    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 20:39:34.627746 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.627730    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal"
Jul 21 20:39:34.645090 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.645070    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 20:39:34.661787 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.661761    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 20:39:34.991655 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:34.991611    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 20:39:35.176428 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.176298    2567 apiserver.go:52] "Watching apiserver"
Jul 21 20:39:35.184704 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.184680    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 20:39:35.186648 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.186620    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-467q6","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal","openshift-multus/multus-6f7nt","openshift-multus/network-metrics-daemon-tms9z","openshift-network-diagnostics/network-check-target-8cksp","openshift-network-operator/iptables-alerter-vzzpb","kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal","openshift-cluster-node-tuning-operator/tuned-s2kfw","openshift-dns/node-resolver-n2h4z","openshift-image-registry/node-ca-lcbgb","openshift-multus/multus-additional-cni-plugins-wc6xm","openshift-ovn-kubernetes/ovnkube-node-pv66t","kube-system/global-pull-secret-syncer-pdlgq"]
Jul 21 20:39:35.188967 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.188952    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.191112 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.191093    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.193394 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.193257    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.193394 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.193317    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:35.195508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.195331    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.197702 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.197686    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:35.197774 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.197751    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:35.199668 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.199585    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:35.199668 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.199639    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:35.199881 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.199824    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 20:39:35.201861 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.201843    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.204040 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.204023    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:35.204784 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.204768    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 20:39:35.205087 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.205023    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 20:39:35.205804 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.205392    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tqf9d\""
Jul 21 20:39:35.206087 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.205995    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 20:39:35.206087 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.206020    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 20:39:35.206219 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.206205    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q8lpq\""
Jul 21 20:39:35.206374 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.206356    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 20:39:35.206484 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.206466    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 20:39:35.206601 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.206583    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 20:39:35.206745 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.206729    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 20:39:35.206894 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.206880    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w5b2x\""
Jul 21 20:39:35.207051 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.207033    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mtcdl\""
Jul 21 20:39:35.207403 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.207050    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 20:39:35.208169 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.208148    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.208437 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.208420    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 20:39:35.208699 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.208681    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 20:39:35.209101 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.209085    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zcb4g\""
Jul 21 20:39:35.209375 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.209361    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 20:39:35.209494 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.209475    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 20:39:35.209937 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.208975    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 20:39:35.210028 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210016    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 20:39:35.210191 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210171    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-dbus\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.210255 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210210    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 20:39:35.210319 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysctl-conf\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.210371 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210351    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-lib-modules\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.210419 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210383    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5plp\" (UniqueName: \"kubernetes.io/projected/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-kube-api-access-f5plp\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.210466 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-var-lib-kubelet\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.210466 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210440    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9f6b1cba-fb90-4b9e-b51d-e0027828097b-iptables-alerter-script\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.210560 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210470    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-os-release\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.210560 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210496    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-host\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.210560 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210539    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.210690 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-cni-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.210690 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210632    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-tuned\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.210690 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-kubelet\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.210690 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210685    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovnkube-config\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.210875 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210728    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-kubelet\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.210875 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-systemd-units\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.210875 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210799    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d87f0ada-c628-4ac1-ad23-a72e782af939-tmp\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.210875 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210842    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-kubelet-config\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.211085 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210870    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysconfig\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.211085 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210895    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-systemd\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.211085 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-run-netns\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.211085 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-socket-dir-parent\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.211085 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.210995    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-conf-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.211085 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.211019    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q4st\" (UniqueName: \"kubernetes.io/projected/9f6b1cba-fb90-4b9e-b51d-e0027828097b-kube-api-access-5q4st\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.211085 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.211048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-var-lib-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.211085 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.211078    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-cni-netd\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.211995 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.211102    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-netns\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.212100 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.211341    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 20:39:35.212225 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.212179    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-daemon-config\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.212288 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.212233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:35.212288 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.212266    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:35.212388 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.212293    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-kubernetes\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.212388 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.212346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-run\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.212480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.212369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj28d\" (UniqueName: \"kubernetes.io/projected/d87f0ada-c628-4ac1-ad23-a72e782af939-kube-api-access-rj28d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.212480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.212440    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6b1cba-fb90-4b9e-b51d-e0027828097b-host-slash\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.212791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.212503    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-etc-kubernetes\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-modprobe-d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213245    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-ovn\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213304    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-cni-bin\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213333    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-env-overrides\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovnkube-script-lib\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysctl-d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-sys\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213453    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovn-node-metrics-cert\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.213481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-slash\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.214049 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-node-log\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.214544 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214453    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 20:39:35.214653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214635    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 20:39:35.214717 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.213562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-cnibin\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.214717 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214689    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-cni-binary-copy\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.214717 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214712    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-k8s-cni-cncf-io\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.214845 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.214845 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214790    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-system-cni-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.214845 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214814    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-hostroot\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.214845 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214843    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-multus-certs\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.215067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214865    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srxvl\" (UniqueName: \"kubernetes.io/projected/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-kube-api-access-srxvl\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:35.215067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-etc-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.215067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214891    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dj6h5\""
Jul 21 20:39:35.215067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214906    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-cni-bin\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.215067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-cni-multus\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.215067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.214967    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-run-ovn-kubernetes\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.215067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.215007    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hqq9\" (UniqueName: \"kubernetes.io/projected/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-kube-api-access-7hqq9\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.215067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.215041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-systemd\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.215386 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.215077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-log-socket\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.215386 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.215355    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 20:39:35.216695 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.216650    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 20:39:35.216777 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.216705    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.218220 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.218204    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t9gtd\""
Jul 21 20:39:35.218297 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.218270    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 20:39:35.219123 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.219106    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.221476 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.221460    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 20:39:35.222128 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.222109    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 20:39:35.222354 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.222330    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 20:39:35.222433 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.222373    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6fqct\""
Jul 21 20:39:35.222580 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.222550    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 20:39:35.222774 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.222757    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 20:39:35.222840 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.222801    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-87c2m\""
Jul 21 20:39:35.223192 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.223175    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 20:39:35.276010 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.275970    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8gsnm"
Jul 21 20:39:35.293371 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.293346    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8gsnm"
Jul 21 20:39:35.301382 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.301363    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 20:39:35.308792 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.308754    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal" event={"ID":"1399074c7686ed18dafa282fd37a44d0","Type":"ContainerStarted","Data":"b334cf30934af329601a945e20fdd4f344824e97b756bf004cb45bb2bdf1bf5e"}
Jul 21 20:39:35.309824 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.309802    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal" event={"ID":"4e9f1b496a2d166fa089beef280b25f7","Type":"ContainerStarted","Data":"7868b9f8d27d87b3085add03b0e6385c4116644129c336865f888f00e9701df6"}
Jul 21 20:39:35.316129 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-socket-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.316129 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316110    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a56b897c-e624-4d27-a9bb-8e36b5121ce2-hosts-file\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.316245 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:35.316245 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:35.316245 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-kubernetes\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.316382 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316252    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-run\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.316382 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316307    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-run\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.316382 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316373    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-kubernetes\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.316487 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.316402    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:35.316519 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.316482    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs podName:cf884bc1-53a5-47ab-98cd-3959ded9e6e7 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:35.816444156 +0000 UTC m=+3.026472558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs") pod "network-metrics-daemon-tms9z" (UID: "cf884bc1-53a5-47ab-98cd-3959ded9e6e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:35.316519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316505    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj28d\" (UniqueName: \"kubernetes.io/projected/d87f0ada-c628-4ac1-ad23-a72e782af939-kube-api-access-rj28d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.316619 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6b1cba-fb90-4b9e-b51d-e0027828097b-host-slash\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.316619 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316547    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-etc-kubernetes\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.316619 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316568    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-modprobe-d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.316619 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316602    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f6b1cba-fb90-4b9e-b51d-e0027828097b-host-slash\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316607    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316651    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316671    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-ovn\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316697    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-cni-bin\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316715    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-ovn\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316721    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-env-overrides\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316727    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-modprobe-d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316615    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-etc-kubernetes\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovnkube-script-lib\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316784    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-cni-bin\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.316818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316805    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysctl-d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316844    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-sys\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316899    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovn-node-metrics-cert\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.316946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh9g5\" (UniqueName: \"kubernetes.io/projected/a56b897c-e624-4d27-a9bb-8e36b5121ce2-kube-api-access-fh9g5\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317001    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-system-cni-dir\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysctl-d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-slash\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317153    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-slash\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-node-log\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317217    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-sys\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-cnibin\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317336    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-env-overrides\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317349    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-cni-binary-copy\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-node-log\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.317379 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317376    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-k8s-cni-cncf-io\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317401    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-cnibin\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317439    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317457    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-k8s-cni-cncf-io\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317539    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovnkube-script-lib\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317557    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317589    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-system-cni-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-hostroot\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317625    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317676    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-hostroot\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317693    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-system-cni-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.317701    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317703    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-multus-certs\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-multus-certs\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317736    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srxvl\" (UniqueName: \"kubernetes.io/projected/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-kube-api-access-srxvl\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.317889    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret podName:cf604c6d-39e1-43a4-8cfc-9f06bf3cd729 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:35.817874045 +0000 UTC m=+3.027902458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret") pod "global-pull-secret-syncer-pdlgq" (UID: "cf604c6d-39e1-43a4-8cfc-9f06bf3cd729") : object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-etc-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-cni-bin\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318002 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.317962    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-cni-multus\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-serviceca\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318009    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-etc-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318027    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-cnibin\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318052    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-os-release\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318067    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-cni-bin\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-run-ovn-kubernetes\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318067    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-cni-binary-copy\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318054    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-cni-multus\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hqq9\" (UniqueName: \"kubernetes.io/projected/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-kube-api-access-7hqq9\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318127    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/37291fdd-4b40-4d1a-af18-99d28c690348-agent-certs\") pod \"konnectivity-agent-467q6\" (UID: \"37291fdd-4b40-4d1a-af18-99d28c690348\") " pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318137    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-run-ovn-kubernetes\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/37291fdd-4b40-4d1a-af18-99d28c690348-konnectivity-ca\") pod \"konnectivity-agent-467q6\" (UID: \"37291fdd-4b40-4d1a-af18-99d28c690348\") " pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rfhd\" (UniqueName: \"kubernetes.io/projected/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-kube-api-access-6rfhd\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-host\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318279    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-systemd\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.318772 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318294    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-log-socket\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-run-systemd\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-dbus\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318380    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-log-socket\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318374    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318421    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysctl-conf\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318443    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-lib-modules\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318515    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5plp\" (UniqueName: \"kubernetes.io/projected/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-kube-api-access-f5plp\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318525    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-dbus\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-lib-modules\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318552    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysctl-conf\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318591    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-var-lib-kubelet\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318631    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9f6b1cba-fb90-4b9e-b51d-e0027828097b-iptables-alerter-script\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318657    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-device-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318669    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-var-lib-kubelet\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318681    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-sys-fs\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.319519 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318704    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318727    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-os-release\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318747    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-host\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318812    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-os-release\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-cni-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318856    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318826    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-host\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318885    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-cni-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318866    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsw4c\" (UniqueName: \"kubernetes.io/projected/d3004918-748e-45ab-85e8-7eefa1d359b3-kube-api-access-wsw4c\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-tuned\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.318946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-kubelet\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovnkube-config\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319036    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-kubelet\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-kubelet\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319062    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-var-lib-kubelet\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319082    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a56b897c-e624-4d27-a9bb-8e36b5121ce2-tmp-dir\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.320325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319118    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319152    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-systemd-units\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319165    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9f6b1cba-fb90-4b9e-b51d-e0027828097b-iptables-alerter-script\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d87f0ada-c628-4ac1-ad23-a72e782af939-tmp\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-systemd-units\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319284    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-kubelet-config\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319313    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-registration-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319339    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysconfig\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319398    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-sysconfig\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319383    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-systemd\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319418    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-kubelet-config\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319434    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-run-netns\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-socket-dir-parent\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-systemd\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319495    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-conf-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319513    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-run-netns\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-socket-dir-parent\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.320909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319553    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjh75\" (UniqueName: \"kubernetes.io/projected/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-kube-api-access-kjh75\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319586    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-conf-dir\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q4st\" (UniqueName: \"kubernetes.io/projected/9f6b1cba-fb90-4b9e-b51d-e0027828097b-kube-api-access-5q4st\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovnkube-config\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-var-lib-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319648    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-cni-netd\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319659    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-var-lib-openvswitch\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319709    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-netns\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-daemon-config\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319767    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-host-cni-netd\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.319816    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-host-run-netns\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.320302    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-multus-daemon-config\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.321345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-ovn-node-metrics-cert\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.321694 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.321557    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d87f0ada-c628-4ac1-ad23-a72e782af939-tmp\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.322305 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.321787    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d87f0ada-c628-4ac1-ad23-a72e782af939-etc-tuned\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.328953 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.328933    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 20:39:35.329054 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.328972    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 20:39:35.329054 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.329007    2567 projected.go:196] Error preparing data for projected volume kube-api-access-hffth for pod openshift-network-diagnostics/network-check-target-8cksp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:35.329164 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.329063    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth podName:6e320cc6-4317-4840-90a4-2d07ff82d19c nodeName:}" failed. No retries permitted until 2026-07-21 20:39:35.829049573 +0000 UTC m=+3.039077983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hffth" (UniqueName: "kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth") pod "network-check-target-8cksp" (UID: "6e320cc6-4317-4840-90a4-2d07ff82d19c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:35.330810 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.330772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srxvl\" (UniqueName: \"kubernetes.io/projected/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-kube-api-access-srxvl\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:35.330889 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.330850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5plp\" (UniqueName: \"kubernetes.io/projected/9c6e732b-dd0c-4b2e-92c3-28daefa0db18-kube-api-access-f5plp\") pod \"ovnkube-node-pv66t\" (UID: \"9c6e732b-dd0c-4b2e-92c3-28daefa0db18\") " pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.331048 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.331030    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj28d\" (UniqueName: \"kubernetes.io/projected/d87f0ada-c628-4ac1-ad23-a72e782af939-kube-api-access-rj28d\") pod \"tuned-s2kfw\" (UID: \"d87f0ada-c628-4ac1-ad23-a72e782af939\") " pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.331416 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.331386    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q4st\" (UniqueName: \"kubernetes.io/projected/9f6b1cba-fb90-4b9e-b51d-e0027828097b-kube-api-access-5q4st\") pod \"iptables-alerter-vzzpb\" (UID: \"9f6b1cba-fb90-4b9e-b51d-e0027828097b\") " pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.331416 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.331386    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hqq9\" (UniqueName: \"kubernetes.io/projected/d8fd5a6d-ebe9-432c-b317-d1ae72fc149b-kube-api-access-7hqq9\") pod \"multus-6f7nt\" (UID: \"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b\") " pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.420329 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-socket-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a56b897c-e624-4d27-a9bb-8e36b5121ce2-hosts-file\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh9g5\" (UniqueName: \"kubernetes.io/projected/a56b897c-e624-4d27-a9bb-8e36b5121ce2-kube-api-access-fh9g5\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420395    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-system-cni-dir\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-serviceca\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420446    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a56b897c-e624-4d27-a9bb-8e36b5121ce2-hosts-file\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-cnibin\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420489    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-socket-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-os-release\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420484    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-system-cni-dir\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.420508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420511    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420520    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/37291fdd-4b40-4d1a-af18-99d28c690348-agent-certs\") pod \"konnectivity-agent-467q6\" (UID: \"37291fdd-4b40-4d1a-af18-99d28c690348\") " pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420536    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/37291fdd-4b40-4d1a-af18-99d28c690348-konnectivity-ca\") pod \"konnectivity-agent-467q6\" (UID: \"37291fdd-4b40-4d1a-af18-99d28c690348\") " pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rfhd\" (UniqueName: \"kubernetes.io/projected/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-kube-api-access-6rfhd\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420549    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-cnibin\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420563    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-host\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420581    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-os-release\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420610    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-host\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420631    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420690    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420714    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-device-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-sys-fs\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420755    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420778    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsw4c\" (UniqueName: \"kubernetes.io/projected/d3004918-748e-45ab-85e8-7eefa1d359b3-kube-api-access-wsw4c\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420801    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a56b897c-e624-4d27-a9bb-8e36b5121ce2-tmp-dir\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420843    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420869    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-registration-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420879    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-sys-fs\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjh75\" (UniqueName: \"kubernetes.io/projected/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-kube-api-access-kjh75\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420870    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3004918-748e-45ab-85e8-7eefa1d359b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420937    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-device-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.420947    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-serviceca\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.421135    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/37291fdd-4b40-4d1a-af18-99d28c690348-konnectivity-ca\") pod \"konnectivity-agent-467q6\" (UID: \"37291fdd-4b40-4d1a-af18-99d28c690348\") " pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.421247    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.421388    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a56b897c-e624-4d27-a9bb-8e36b5121ce2-tmp-dir\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.421404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.421443    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-registration-dir\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.421776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.421477    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.422190 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.421846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3004918-748e-45ab-85e8-7eefa1d359b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.423073 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.423051    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/37291fdd-4b40-4d1a-af18-99d28c690348-agent-certs\") pod \"konnectivity-agent-467q6\" (UID: \"37291fdd-4b40-4d1a-af18-99d28c690348\") " pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:35.429727 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.429693    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rfhd\" (UniqueName: \"kubernetes.io/projected/cb69f95a-4acc-4698-aa9d-e990e7ae25a1-kube-api-access-6rfhd\") pod \"aws-ebs-csi-driver-node-65q7r\" (UID: \"cb69f95a-4acc-4698-aa9d-e990e7ae25a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.430297 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.430265    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsw4c\" (UniqueName: \"kubernetes.io/projected/d3004918-748e-45ab-85e8-7eefa1d359b3-kube-api-access-wsw4c\") pod \"multus-additional-cni-plugins-wc6xm\" (UID: \"d3004918-748e-45ab-85e8-7eefa1d359b3\") " pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.430472 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.430453    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh9g5\" (UniqueName: \"kubernetes.io/projected/a56b897c-e624-4d27-a9bb-8e36b5121ce2-kube-api-access-fh9g5\") pod \"node-resolver-n2h4z\" (UID: \"a56b897c-e624-4d27-a9bb-8e36b5121ce2\") " pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.430536 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.430460    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjh75\" (UniqueName: \"kubernetes.io/projected/33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb-kube-api-access-kjh75\") pod \"node-ca-lcbgb\" (UID: \"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb\") " pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.502017 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.501928    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-vzzpb"
Jul 21 20:39:35.509801 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.509779    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:39:35.518282 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.518262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6f7nt"
Jul 21 20:39:35.524385 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.524359    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s2kfw"
Jul 21 20:39:35.531937 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.531920    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:35.538301 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.538281    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r"
Jul 21 20:39:35.546780 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.546763    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n2h4z"
Jul 21 20:39:35.552202 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.552183    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lcbgb"
Jul 21 20:39:35.557511 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.557492    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wc6xm"
Jul 21 20:39:35.823888 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.823806    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:35.823888 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.823861    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:35.824101 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.823960    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:35.824101 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.823967    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:35.824101 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.824033    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs podName:cf884bc1-53a5-47ab-98cd-3959ded9e6e7 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:36.824016752 +0000 UTC m=+4.034045162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs") pod "network-metrics-daemon-tms9z" (UID: "cf884bc1-53a5-47ab-98cd-3959ded9e6e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:35.824101 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.824049    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret podName:cf604c6d-39e1-43a4-8cfc-9f06bf3cd729 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:36.824041975 +0000 UTC m=+4.034070378 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret") pod "global-pull-secret-syncer-pdlgq" (UID: "cf604c6d-39e1-43a4-8cfc-9f06bf3cd729") : object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:35.924228 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:35.924194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:35.924418 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.924398    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 20:39:35.924457 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.924426    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 20:39:35.924457 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.924439    2567 projected.go:196] Error preparing data for projected volume kube-api-access-hffth for pod openshift-network-diagnostics/network-check-target-8cksp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:35.924517 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:35.924501    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth podName:6e320cc6-4317-4840-90a4-2d07ff82d19c nodeName:}" failed. No retries permitted until 2026-07-21 20:39:36.924483926 +0000 UTC m=+4.134512335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffth" (UniqueName: "kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth") pod "network-check-target-8cksp" (UID: "6e320cc6-4317-4840-90a4-2d07ff82d19c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:36.224297 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.224264    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd87f0ada_c628_4ac1_ad23_a72e782af939.slice/crio-25ed5656f36510db68a8e932813c63b83459c8c940121a0e12f09cfdc9347a09 WatchSource:0}: Error finding container 25ed5656f36510db68a8e932813c63b83459c8c940121a0e12f09cfdc9347a09: Status 404 returned error can't find the container with id 25ed5656f36510db68a8e932813c63b83459c8c940121a0e12f09cfdc9347a09
Jul 21 20:39:36.225150 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.225115    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8fd5a6d_ebe9_432c_b317_d1ae72fc149b.slice/crio-12727edc4659d07f60403f98d5abfb40accd2f9558fe7c39db45b92a30468cf6 WatchSource:0}: Error finding container 12727edc4659d07f60403f98d5abfb40accd2f9558fe7c39db45b92a30468cf6: Status 404 returned error can't find the container with id 12727edc4659d07f60403f98d5abfb40accd2f9558fe7c39db45b92a30468cf6
Jul 21 20:39:36.228649 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.228609    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3004918_748e_45ab_85e8_7eefa1d359b3.slice/crio-34f6ac0d86545281aafaf96e5139d925e55aee77c48e826e7e4de237e285b539 WatchSource:0}: Error finding container 34f6ac0d86545281aafaf96e5139d925e55aee77c48e826e7e4de237e285b539: Status 404 returned error can't find the container with id 34f6ac0d86545281aafaf96e5139d925e55aee77c48e826e7e4de237e285b539
Jul 21 20:39:36.230575 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.230558    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c6e732b_dd0c_4b2e_92c3_28daefa0db18.slice/crio-0a554c65799a63ba6472a7eb3794609ac85815e29b1f6c36c11d694f19e62601 WatchSource:0}: Error finding container 0a554c65799a63ba6472a7eb3794609ac85815e29b1f6c36c11d694f19e62601: Status 404 returned error can't find the container with id 0a554c65799a63ba6472a7eb3794609ac85815e29b1f6c36c11d694f19e62601
Jul 21 20:39:36.232039 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.232017    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b3e3b1_3e9e_4ed9_9aa4_68038f31d9fb.slice/crio-904f2ff4b03b219aff7186f5f59cab8634d42682ea4020722aaf15e7e318e353 WatchSource:0}: Error finding container 904f2ff4b03b219aff7186f5f59cab8634d42682ea4020722aaf15e7e318e353: Status 404 returned error can't find the container with id 904f2ff4b03b219aff7186f5f59cab8634d42682ea4020722aaf15e7e318e353
Jul 21 20:39:36.233026 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.233004    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37291fdd_4b40_4d1a_af18_99d28c690348.slice/crio-86c6ed75115d4add3e4a9acc5664e0e19cb367a67d03f317dbd4c44d4229f671 WatchSource:0}: Error finding container 86c6ed75115d4add3e4a9acc5664e0e19cb367a67d03f317dbd4c44d4229f671: Status 404 returned error can't find the container with id 86c6ed75115d4add3e4a9acc5664e0e19cb367a67d03f317dbd4c44d4229f671
Jul 21 20:39:36.234474 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.234322    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6b1cba_fb90_4b9e_b51d_e0027828097b.slice/crio-9a4a599e2eeca8870d14b67329e72f9df1980f18ff632855dda13df525e7668f WatchSource:0}: Error finding container 9a4a599e2eeca8870d14b67329e72f9df1980f18ff632855dda13df525e7668f: Status 404 returned error can't find the container with id 9a4a599e2eeca8870d14b67329e72f9df1980f18ff632855dda13df525e7668f
Jul 21 20:39:36.236221 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.235969    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb69f95a_4acc_4698_aa9d_e990e7ae25a1.slice/crio-45e02d6845eca62531fa54b31c35143feaa2119f92aa4f0ea315cb2de216cf57 WatchSource:0}: Error finding container 45e02d6845eca62531fa54b31c35143feaa2119f92aa4f0ea315cb2de216cf57: Status 404 returned error can't find the container with id 45e02d6845eca62531fa54b31c35143feaa2119f92aa4f0ea315cb2de216cf57
Jul 21 20:39:36.237141 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:39:36.237098    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda56b897c_e624_4d27_a9bb_8e36b5121ce2.slice/crio-fd1d5f77096ce7420353fdf43758e961b372d11e8befbcaa5cd1716a1b827a2e WatchSource:0}: Error finding container fd1d5f77096ce7420353fdf43758e961b372d11e8befbcaa5cd1716a1b827a2e: Status 404 returned error can't find the container with id fd1d5f77096ce7420353fdf43758e961b372d11e8befbcaa5cd1716a1b827a2e
Jul 21 20:39:36.295831 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.295794    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 20:34:35 +0000 UTC" deadline="2028-02-14 19:29:59.003508648 +0000 UTC"
Jul 21 20:39:36.295922 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.295832    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13750h50m22.707679207s"
Jul 21 20:39:36.313940 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.313677    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"0a554c65799a63ba6472a7eb3794609ac85815e29b1f6c36c11d694f19e62601"}
Jul 21 20:39:36.315017 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.314970    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6f7nt" event={"ID":"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b","Type":"ContainerStarted","Data":"12727edc4659d07f60403f98d5abfb40accd2f9558fe7c39db45b92a30468cf6"}
Jul 21 20:39:36.316034 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.316014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s2kfw" event={"ID":"d87f0ada-c628-4ac1-ad23-a72e782af939","Type":"ContainerStarted","Data":"25ed5656f36510db68a8e932813c63b83459c8c940121a0e12f09cfdc9347a09"}
Jul 21 20:39:36.316967 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.316937    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vzzpb" event={"ID":"9f6b1cba-fb90-4b9e-b51d-e0027828097b","Type":"ContainerStarted","Data":"9a4a599e2eeca8870d14b67329e72f9df1980f18ff632855dda13df525e7668f"}
Jul 21 20:39:36.317800 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.317777    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-467q6" event={"ID":"37291fdd-4b40-4d1a-af18-99d28c690348","Type":"ContainerStarted","Data":"86c6ed75115d4add3e4a9acc5664e0e19cb367a67d03f317dbd4c44d4229f671"}
Jul 21 20:39:36.318829 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.318810    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" event={"ID":"d3004918-748e-45ab-85e8-7eefa1d359b3","Type":"ContainerStarted","Data":"34f6ac0d86545281aafaf96e5139d925e55aee77c48e826e7e4de237e285b539"}
Jul 21 20:39:36.320513 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.320483    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal" event={"ID":"1399074c7686ed18dafa282fd37a44d0","Type":"ContainerStarted","Data":"9fa6447674f9da167ada98d1b5cf92155fe73b0f8214ca8d3f031a0cea732e27"}
Jul 21 20:39:36.321499 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.321482    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n2h4z" event={"ID":"a56b897c-e624-4d27-a9bb-8e36b5121ce2","Type":"ContainerStarted","Data":"fd1d5f77096ce7420353fdf43758e961b372d11e8befbcaa5cd1716a1b827a2e"}
Jul 21 20:39:36.322601 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.322570    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r" event={"ID":"cb69f95a-4acc-4698-aa9d-e990e7ae25a1","Type":"ContainerStarted","Data":"45e02d6845eca62531fa54b31c35143feaa2119f92aa4f0ea315cb2de216cf57"}
Jul 21 20:39:36.323790 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.323769    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lcbgb" event={"ID":"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb","Type":"ContainerStarted","Data":"904f2ff4b03b219aff7186f5f59cab8634d42682ea4020722aaf15e7e318e353"}
Jul 21 20:39:36.334575 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.334528    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-177.ec2.internal" podStartSLOduration=2.334516647 podStartE2EDuration="2.334516647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:36.334419723 +0000 UTC m=+3.544448158" watchObservedRunningTime="2026-07-21 20:39:36.334516647 +0000 UTC m=+3.544545064"
Jul 21 20:39:36.832803 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.832201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:36.832803 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.832261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:36.832803 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:36.832367    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:36.832803 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:36.832425    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret podName:cf604c6d-39e1-43a4-8cfc-9f06bf3cd729 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:38.832408504 +0000 UTC m=+6.042436916 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret") pod "global-pull-secret-syncer-pdlgq" (UID: "cf604c6d-39e1-43a4-8cfc-9f06bf3cd729") : object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:36.832803 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:36.832365    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:36.832803 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:36.832495    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs podName:cf884bc1-53a5-47ab-98cd-3959ded9e6e7 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:38.832476923 +0000 UTC m=+6.042505334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs") pod "network-metrics-daemon-tms9z" (UID: "cf884bc1-53a5-47ab-98cd-3959ded9e6e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:36.933558 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:36.933520    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:36.933742 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:36.933696    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 20:39:36.933742 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:36.933715    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 20:39:36.933742 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:36.933728    2567 projected.go:196] Error preparing data for projected volume kube-api-access-hffth for pod openshift-network-diagnostics/network-check-target-8cksp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:36.934127 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:36.933785    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth podName:6e320cc6-4317-4840-90a4-2d07ff82d19c nodeName:}" failed. No retries permitted until 2026-07-21 20:39:38.933766856 +0000 UTC m=+6.143795268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffth" (UniqueName: "kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth") pod "network-check-target-8cksp" (UID: "6e320cc6-4317-4840-90a4-2d07ff82d19c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:37.296942 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:37.296872    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 20:34:35 +0000 UTC" deadline="2028-01-01 11:04:45.28476835 +0000 UTC"
Jul 21 20:39:37.296942 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:37.296909    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12686h25m7.987864468s"
Jul 21 20:39:37.306576 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:37.306117    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:37.306576 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:37.306118    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:37.306576 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:37.306233    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:37.306576 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:37.306299    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:37.306576 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:37.306398    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:37.306576 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:37.306487    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:37.335685 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:37.335652    2567 generic.go:356] "Generic (PLEG): container finished" podID="4e9f1b496a2d166fa089beef280b25f7" containerID="4f12cacee75a529c5001b00e5c81c16c2b046fc1f49f4b82f46b66a7a807c109" exitCode=0
Jul 21 20:39:37.336911 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:37.336882    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal" event={"ID":"4e9f1b496a2d166fa089beef280b25f7","Type":"ContainerDied","Data":"4f12cacee75a529c5001b00e5c81c16c2b046fc1f49f4b82f46b66a7a807c109"}
Jul 21 20:39:38.368057 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:38.367160    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal" event={"ID":"4e9f1b496a2d166fa089beef280b25f7","Type":"ContainerStarted","Data":"9c606b8f819415ec6247bd941856fc4b49e1329eb54cdaea689897c6b7c9dc33"}
Jul 21 20:39:38.387604 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:38.387552    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-177.ec2.internal" podStartSLOduration=4.3875296630000005 podStartE2EDuration="4.387529663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:38.387140888 +0000 UTC m=+5.597169327" watchObservedRunningTime="2026-07-21 20:39:38.387529663 +0000 UTC m=+5.597558089"
Jul 21 20:39:38.849677 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:38.849640    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:38.849870 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:38.849695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:38.849870 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:38.849827    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:38.849981 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:38.849883    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret podName:cf604c6d-39e1-43a4-8cfc-9f06bf3cd729 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:42.849866024 +0000 UTC m=+10.059894433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret") pod "global-pull-secret-syncer-pdlgq" (UID: "cf604c6d-39e1-43a4-8cfc-9f06bf3cd729") : object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:38.850107 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:38.850089    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:38.850177 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:38.850147    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs podName:cf884bc1-53a5-47ab-98cd-3959ded9e6e7 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:42.850132816 +0000 UTC m=+10.060161229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs") pod "network-metrics-daemon-tms9z" (UID: "cf884bc1-53a5-47ab-98cd-3959ded9e6e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:38.950445 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:38.950399    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:38.950709 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:38.950625    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 20:39:38.950709 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:38.950648    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 20:39:38.950709 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:38.950661    2567 projected.go:196] Error preparing data for projected volume kube-api-access-hffth for pod openshift-network-diagnostics/network-check-target-8cksp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:38.950880 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:38.950719    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth podName:6e320cc6-4317-4840-90a4-2d07ff82d19c nodeName:}" failed. No retries permitted until 2026-07-21 20:39:42.950701565 +0000 UTC m=+10.160729978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffth" (UniqueName: "kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth") pod "network-check-target-8cksp" (UID: "6e320cc6-4317-4840-90a4-2d07ff82d19c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:39.305427 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:39.305391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:39.305608 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:39.305524    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:39.305608 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:39.305391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:39.305714 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:39.305635    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:39.306021 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:39.305842    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:39.306021 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:39.305959    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:41.305430 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:41.305232    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:41.305430 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:41.305238    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:41.305430 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:41.305374    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:41.305963 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:41.305449    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:41.305963 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:41.305497    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:41.305963 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:41.305571    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:42.885747 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:42.885691    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:42.886234 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:42.885780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:42.886234 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:42.885906    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:42.886234 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:42.885970    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs podName:cf884bc1-53a5-47ab-98cd-3959ded9e6e7 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:50.885950019 +0000 UTC m=+18.095978432 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs") pod "network-metrics-daemon-tms9z" (UID: "cf884bc1-53a5-47ab-98cd-3959ded9e6e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:42.886395 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:42.886385    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:42.886448 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:42.886436    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret podName:cf604c6d-39e1-43a4-8cfc-9f06bf3cd729 nodeName:}" failed. No retries permitted until 2026-07-21 20:39:50.886420682 +0000 UTC m=+18.096449096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret") pod "global-pull-secret-syncer-pdlgq" (UID: "cf604c6d-39e1-43a4-8cfc-9f06bf3cd729") : object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:42.986563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:42.986524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:42.986766 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:42.986726    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 20:39:42.986766 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:42.986754    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 20:39:42.986766 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:42.986771    2567 projected.go:196] Error preparing data for projected volume kube-api-access-hffth for pod openshift-network-diagnostics/network-check-target-8cksp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:42.986978 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:42.986835    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth podName:6e320cc6-4317-4840-90a4-2d07ff82d19c nodeName:}" failed. No retries permitted until 2026-07-21 20:39:50.986817567 +0000 UTC m=+18.196845971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffth" (UniqueName: "kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth") pod "network-check-target-8cksp" (UID: "6e320cc6-4317-4840-90a4-2d07ff82d19c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:43.307804 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:43.307108    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:43.307804 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:43.307287    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:43.308228 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:43.307821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:43.308228 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:43.307918    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:43.308228 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:43.307972    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:43.308228 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:43.308084    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:45.305051 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:45.305013    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:45.305051 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:45.305038    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:45.305646 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:45.305059    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:45.305646 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:45.305150    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:45.305749 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:45.305645    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:45.305749 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:45.305729    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:47.305485 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:47.305445    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:47.305485 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:47.305479    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:47.306034 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:47.305445    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:47.306034 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:47.305600    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:47.306034 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:47.305722    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:47.306034 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:47.305819    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:49.305590 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:49.305552    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:49.305590 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:49.305586    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:49.306106 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:49.305553    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:49.306106 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:49.305680    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:49.306106 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:49.305740    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:49.306106 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:49.305830    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:50.940118 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:50.940080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:50.940607 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:50.940150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:50.940607 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:50.940262    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:50.940607 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:50.940268    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:50.940607 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:50.940319    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs podName:cf884bc1-53a5-47ab-98cd-3959ded9e6e7 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:06.940304579 +0000 UTC m=+34.150332978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs") pod "network-metrics-daemon-tms9z" (UID: "cf884bc1-53a5-47ab-98cd-3959ded9e6e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:39:50.940607 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:50.940332    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret podName:cf604c6d-39e1-43a4-8cfc-9f06bf3cd729 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:06.940326522 +0000 UTC m=+34.150354921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret") pod "global-pull-secret-syncer-pdlgq" (UID: "cf604c6d-39e1-43a4-8cfc-9f06bf3cd729") : object "kube-system"/"original-pull-secret" not registered
Jul 21 20:39:51.040800 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:51.040759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:51.040972 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:51.040940    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 20:39:51.040972 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:51.040961    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 20:39:51.040972 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:51.040969    2567 projected.go:196] Error preparing data for projected volume kube-api-access-hffth for pod openshift-network-diagnostics/network-check-target-8cksp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:51.041125 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:51.041035    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth podName:6e320cc6-4317-4840-90a4-2d07ff82d19c nodeName:}" failed. No retries permitted until 2026-07-21 20:40:07.041022615 +0000 UTC m=+34.251051018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffth" (UniqueName: "kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth") pod "network-check-target-8cksp" (UID: "6e320cc6-4317-4840-90a4-2d07ff82d19c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:39:51.305852 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:51.305764    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:51.306036 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:51.305764    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:51.306036 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:51.305908    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:51.306036 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:51.305924    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:51.306190 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:51.306068    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:51.306190 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:51.306153    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:53.306417 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:53.306389    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:53.306839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:53.306482    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:53.306839 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:53.306511    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:53.306839 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:53.306567    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:53.306839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:53.306577    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:53.306839 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:53.306695    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:55.306210 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.305540    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:55.306210 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.305565    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:55.306210 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.305595    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:55.306210 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:55.305900    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:55.306933 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:55.306312    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:55.306933 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:55.306406    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:55.404556 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.404529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n2h4z" event={"ID":"a56b897c-e624-4d27-a9bb-8e36b5121ce2","Type":"ContainerStarted","Data":"14da58e82882d646d3c530de90782bc60d041cf471bcefd0eea8983fab93a0cc"}
Jul 21 20:39:55.405894 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.405872    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r" event={"ID":"cb69f95a-4acc-4698-aa9d-e990e7ae25a1","Type":"ContainerStarted","Data":"0905fe0b97f3ec00b641d4165cef8bbf39dc6277b3e8962bc67ae42029327f1a"}
Jul 21 20:39:55.407137 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.407115    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lcbgb" event={"ID":"33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb","Type":"ContainerStarted","Data":"ea7e62660304b39930da88c4c232e912b6f488d9e8a86dd3171d143e25d979ac"}
Jul 21 20:39:55.408463 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.408442    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"d4706f887722d442e135e0fdae61539a573586e857f8d2b1edda54ed703df6a1"}
Jul 21 20:39:55.409837 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.409815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6f7nt" event={"ID":"d8fd5a6d-ebe9-432c-b317-d1ae72fc149b","Type":"ContainerStarted","Data":"d942bdb77f8d5c8211ee9a8e6e3493ac23d7468ffa765037bcb2e4e8f4387496"}
Jul 21 20:39:55.411130 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.411113    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s2kfw" event={"ID":"d87f0ada-c628-4ac1-ad23-a72e782af939","Type":"ContainerStarted","Data":"d1da86026c849b27c7755aac26b2537eb747e331aeab40c3beb114f0c48926ca"}
Jul 21 20:39:55.412400 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.412384    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-467q6" event={"ID":"37291fdd-4b40-4d1a-af18-99d28c690348","Type":"ContainerStarted","Data":"81d7eacae017d3f5dde57765a3ac902b357b2152033d200a2c38506c5183e046"}
Jul 21 20:39:55.413812 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.413791    2567 generic.go:356] "Generic (PLEG): container finished" podID="d3004918-748e-45ab-85e8-7eefa1d359b3" containerID="f2ce185560b86e40f7210b394d1427d3783e24ad509b6d46341aa78694b9b87f" exitCode=0
Jul 21 20:39:55.413899 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.413827    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" event={"ID":"d3004918-748e-45ab-85e8-7eefa1d359b3","Type":"ContainerDied","Data":"f2ce185560b86e40f7210b394d1427d3783e24ad509b6d46341aa78694b9b87f"}
Jul 21 20:39:55.419828 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.419793    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n2h4z" podStartSLOduration=3.884624617 podStartE2EDuration="22.419782522s" totalImagesPullingTime="18.535157905s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:55.41961854 +0000 UTC m=+22.629646986" watchObservedRunningTime="2026-07-21 20:39:55.419782522 +0000 UTC m=+22.629810942"
Jul 21 20:39:55.464643 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.464596    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6f7nt" podStartSLOduration=3.8599428749999998 podStartE2EDuration="22.464581209s" totalImagesPullingTime="18.604638334s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:55.464283762 +0000 UTC m=+22.674312204" watchObservedRunningTime="2026-07-21 20:39:55.464581209 +0000 UTC m=+22.674609629"
Jul 21 20:39:55.481303 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.481262    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s2kfw" podStartSLOduration=3.937951853 podStartE2EDuration="22.481248925s" totalImagesPullingTime="18.543297072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:55.480871694 +0000 UTC m=+22.690900141" watchObservedRunningTime="2026-07-21 20:39:55.481248925 +0000 UTC m=+22.691277368"
Jul 21 20:39:55.497810 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.497769    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-467q6" podStartSLOduration=4.054599353 podStartE2EDuration="22.497753219s" totalImagesPullingTime="18.443153866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:55.497488551 +0000 UTC m=+22.707516992" watchObservedRunningTime="2026-07-21 20:39:55.497753219 +0000 UTC m=+22.707781639"
Jul 21 20:39:55.527271 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:55.527232    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lcbgb" podStartSLOduration=4.024861168 podStartE2EDuration="22.527215528s" totalImagesPullingTime="18.50235436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:55.527161987 +0000 UTC m=+22.737190405" watchObservedRunningTime="2026-07-21 20:39:55.527215528 +0000 UTC m=+22.737243931"
Jul 21 20:39:56.419298 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.419271    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:39:56.420063 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.419674    2567 generic.go:356] "Generic (PLEG): container finished" podID="9c6e732b-dd0c-4b2e-92c3-28daefa0db18" containerID="643aaef4b34f77c397b523eb83f5249f73b7db354eee2604cf5ec66c354f7282" exitCode=1
Jul 21 20:39:56.420063 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.419707    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"aff52c156ee23236682c0292608fcb38789e01157d39c98fc23ad10ec45f7f4d"}
Jul 21 20:39:56.420063 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.419824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"85da844bee1e7959459a588ce296a2e1979a52d6a020fd410e04f3b693287998"}
Jul 21 20:39:56.420063 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.419875    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"03b9cf318082b4a5f3c2a14fc240cea3e61083e72414364d9940ea88fdc3ccae"}
Jul 21 20:39:56.420063 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.419924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"6214a9acd6de5fb80253b0e9a0b201f522cc431e53d439fb92f75bbaa9a373d7"}
Jul 21 20:39:56.420063 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.419973    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerDied","Data":"643aaef4b34f77c397b523eb83f5249f73b7db354eee2604cf5ec66c354f7282"}
Jul 21 20:39:56.422898 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.422867    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-vzzpb" event={"ID":"9f6b1cba-fb90-4b9e-b51d-e0027828097b","Type":"ContainerStarted","Data":"253c6266eb50c9abe82d777863f1d270a70537b548850046ba77d2a0230a2d92"}
Jul 21 20:39:56.604784 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:56.604710    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 20:39:57.152875 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.152839    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:57.153776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.153591    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:57.168973 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.168920    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-vzzpb" podStartSLOduration=5.638056739 podStartE2EDuration="24.168898544s" totalImagesPullingTime="18.530841805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:56.438293514 +0000 UTC m=+23.648321964" watchObservedRunningTime="2026-07-21 20:39:57.168898544 +0000 UTC m=+24.378926967"
Jul 21 20:39:57.270382 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.270288    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T20:39:56.60473498Z","UUID":"d0c7f114-c82c-4fbb-ac60-c15c935b9324","Handler":null,"Name":"","Endpoint":""}
Jul 21 20:39:57.272119 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.272098    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 20:39:57.272247 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.272136    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 20:39:57.305642 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.305613    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:57.305816 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:57.305743    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:39:57.305886 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.305838    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:57.305966 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:57.305934    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:57.306130 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.305976    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:57.306130 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:57.306092    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:57.426882 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.426819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r" event={"ID":"cb69f95a-4acc-4698-aa9d-e990e7ae25a1","Type":"ContainerStarted","Data":"fd48ddb1e6ddc37cb65136a9f7a480d96ee43f5238cfd7ccf67d367f569973aa"}
Jul 21 20:39:57.427466 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.427369    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:57.428048 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:57.428025    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-467q6"
Jul 21 20:39:58.431072 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:58.430829    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r" event={"ID":"cb69f95a-4acc-4698-aa9d-e990e7ae25a1","Type":"ContainerStarted","Data":"e16e9da0583829ac816fbf667eb956d40f9ead6ee6b33ea21d5ff3df074d1c8f"}
Jul 21 20:39:58.436425 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:58.436405    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:39:58.436827 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:58.436806    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"52189e344433facab528375615ef5a6fd63998d11d376b223529affe90e6901c"}
Jul 21 20:39:58.462549 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:58.462504    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-65q7r" podStartSLOduration=4.673777126 podStartE2EDuration="25.462489712s" totalImagesPullingTime="20.788712586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:39:58.461623751 +0000 UTC m=+25.671652183" watchObservedRunningTime="2026-07-21 20:39:58.462489712 +0000 UTC m=+25.672518132"
Jul 21 20:39:59.305537 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:59.305492    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:39:59.305537 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:59.305519    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:39:59.305813 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:39:59.305613    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:39:59.305813 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:59.305627    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:39:59.305813 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:59.305753    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:39:59.305962 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:39:59.305832    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:40:00.443248 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:00.443218    2567 generic.go:356] "Generic (PLEG): container finished" podID="d3004918-748e-45ab-85e8-7eefa1d359b3" containerID="87c12495704cbc4534d6d3bf072ddc8ef58b3a8335a8058c42f529ac48811eef" exitCode=0
Jul 21 20:40:00.443848 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:00.443253    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" event={"ID":"d3004918-748e-45ab-85e8-7eefa1d359b3","Type":"ContainerDied","Data":"87c12495704cbc4534d6d3bf072ddc8ef58b3a8335a8058c42f529ac48811eef"}
Jul 21 20:40:01.305148 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.305103    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:01.305350 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.305112    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:01.305350 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:01.305242    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:40:01.305476 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:01.305348    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:40:01.305476 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.305112    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:40:01.305566 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:01.305491    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:40:01.448269 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.448240    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:40:01.448852 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.448581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"cb14e3de6a84b6c032686a0c0a7ff84db02f2521ea34b8c8a9b522d71d666f9c"}
Jul 21 20:40:01.448852 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.448836    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:40:01.448967 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.448900    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:40:01.449054 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.449038    2567 scope.go:117] "RemoveContainer" containerID="643aaef4b34f77c397b523eb83f5249f73b7db354eee2604cf5ec66c354f7282"
Jul 21 20:40:01.464912 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:01.464894    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:40:02.454565 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.454538    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:40:02.455110 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.454923    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" event={"ID":"9c6e732b-dd0c-4b2e-92c3-28daefa0db18","Type":"ContainerStarted","Data":"15fae65eaf38b4d6314754d89eab5a70b9c8a0e364b9b301281a627aaa528a05"}
Jul 21 20:40:02.455237 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.455215    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:40:02.457210 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.457187    2567 generic.go:356] "Generic (PLEG): container finished" podID="d3004918-748e-45ab-85e8-7eefa1d359b3" containerID="e4fb34b782dad2a24eca339265af80d0146233e7e0e78422c23a2ba2319c1fa0" exitCode=0
Jul 21 20:40:02.457332 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.457216    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" event={"ID":"d3004918-748e-45ab-85e8-7eefa1d359b3","Type":"ContainerDied","Data":"e4fb34b782dad2a24eca339265af80d0146233e7e0e78422c23a2ba2319c1fa0"}
Jul 21 20:40:02.471814 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.471789    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:40:02.523440 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.523390    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t" podStartSLOduration=10.447696443 podStartE2EDuration="29.523373301s" totalImagesPullingTime="19.075676858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:02.523118686 +0000 UTC m=+29.733147106" watchObservedRunningTime="2026-07-21 20:40:02.523373301 +0000 UTC m=+29.733401723"
Jul 21 20:40:02.523802 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.523788    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pdlgq"]
Jul 21 20:40:02.523891 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.523880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:02.523998 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:02.523966    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:40:02.528841 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.528820    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tms9z"]
Jul 21 20:40:02.528954 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.528942    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:40:02.529058 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:02.529040    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:40:02.545673 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.545653    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8cksp"]
Jul 21 20:40:02.545785 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:02.545741    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:02.545856 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:02.545835    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:40:04.305306 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:04.305277    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:04.305838 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:04.305282    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:04.305838 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:04.305378    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:40:04.305838 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:04.305282    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:40:04.305838 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:04.305453    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:40:04.305838 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:04.305526    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:40:04.464368 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:04.464332    2567 generic.go:356] "Generic (PLEG): container finished" podID="d3004918-748e-45ab-85e8-7eefa1d359b3" containerID="ea6cdf6fb29bdc2a381c1f25bc049ab9fab6eb2f6fb2677b2ec079edb01fd610" exitCode=0
Jul 21 20:40:04.464559 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:04.464377    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" event={"ID":"d3004918-748e-45ab-85e8-7eefa1d359b3","Type":"ContainerDied","Data":"ea6cdf6fb29bdc2a381c1f25bc049ab9fab6eb2f6fb2677b2ec079edb01fd610"}
Jul 21 20:40:06.305557 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:06.305524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:06.306118 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:06.305560    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:06.306118 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:06.305524    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:40:06.306118 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:06.305637    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:40:06.306118 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:06.305712    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:40:06.306118 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:06.305788    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:40:06.961050 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:06.961010    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:40:06.961229 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:06.961060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:06.961229 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:06.961183    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 20:40:06.961341 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:06.961239    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret podName:cf604c6d-39e1-43a4-8cfc-9f06bf3cd729 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:38.961223512 +0000 UTC m=+66.171251931 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret") pod "global-pull-secret-syncer-pdlgq" (UID: "cf604c6d-39e1-43a4-8cfc-9f06bf3cd729") : object "kube-system"/"original-pull-secret" not registered
Jul 21 20:40:06.961341 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:06.961185    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:40:06.961341 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:06.961334    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs podName:cf884bc1-53a5-47ab-98cd-3959ded9e6e7 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:38.961315008 +0000 UTC m=+66.171343441 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs") pod "network-metrics-daemon-tms9z" (UID: "cf884bc1-53a5-47ab-98cd-3959ded9e6e7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 20:40:07.061371 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:07.061338    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:07.061511 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:07.061492    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 20:40:07.061511 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:07.061510    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 20:40:07.061576 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:07.061519    2567 projected.go:196] Error preparing data for projected volume kube-api-access-hffth for pod openshift-network-diagnostics/network-check-target-8cksp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:40:07.061576 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:07.061570    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth podName:6e320cc6-4317-4840-90a4-2d07ff82d19c nodeName:}" failed. No retries permitted until 2026-07-21 20:40:39.061555246 +0000 UTC m=+66.271583652 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hffth" (UniqueName: "kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth") pod "network-check-target-8cksp" (UID: "6e320cc6-4317-4840-90a4-2d07ff82d19c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 20:40:08.305478 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.305393    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:40:08.305478 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.305414    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:08.306029 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.305526    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:08.306029 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:08.305537    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tms9z" podUID="cf884bc1-53a5-47ab-98cd-3959ded9e6e7"
Jul 21 20:40:08.306029 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:08.305632    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8cksp" podUID="6e320cc6-4317-4840-90a4-2d07ff82d19c"
Jul 21 20:40:08.306029 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:08.305698    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pdlgq" podUID="cf604c6d-39e1-43a4-8cfc-9f06bf3cd729"
Jul 21 20:40:08.537566 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.537326    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-177.ec2.internal" event="NodeReady"
Jul 21 20:40:08.537730 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.537708    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 20:40:08.592311 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.592233    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-59b8cd454d-pxwjg"]
Jul 21 20:40:08.595087 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.595062    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.598728 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.598700    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 20:40:08.599791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.599771    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 20:40:08.600008 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.599976    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 20:40:08.600089 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.599977    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9scjn\""
Jul 21 20:40:08.605130 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.605111    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 20:40:08.614624 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.614596    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59b8cd454d-pxwjg"]
Jul 21 20:40:08.618674 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.618629    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7qx6q"]
Jul 21 20:40:08.621058 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.621026    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p5j5p"]
Jul 21 20:40:08.621180 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.621167    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.623910 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.623880    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p5j5p"
Jul 21 20:40:08.624809 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.624593    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 20:40:08.624809 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.624627    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lm6sh\""
Jul 21 20:40:08.624946 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.624910    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 20:40:08.626321 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.626303    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 20:40:08.629071 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.629053    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 20:40:08.629205 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.629188    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 20:40:08.629281 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.629263    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qq4pr\""
Jul 21 20:40:08.640972 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.640951    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7qx6q"]
Jul 21 20:40:08.651342 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.651319    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p5j5p"]
Jul 21 20:40:08.674994 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.674966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-trusted-ca\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.675118 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.675031    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-installation-pull-secrets\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.675118 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.675062    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbst2\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-kube-api-access-sbst2\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.675225 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.675168    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ff55187b-7adc-4356-84cb-8c3830f02ed7-ca-trust-extracted\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.675281 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.675221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.675281 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.675259    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-image-registry-private-configuration\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.675377 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.675281    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-certificates\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.675377 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.675310    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-bound-sa-token\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.776489 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776446    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.776489 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776487    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-image-registry-private-configuration\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.776717 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:08.776598    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 20:40:08.776717 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:08.776619    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b8cd454d-pxwjg: secret "image-registry-tls" not found
Jul 21 20:40:08.776717 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776630    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-certificates\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.776717 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:08.776678    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls podName:ff55187b-7adc-4356-84cb-8c3830f02ed7 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:09.276658226 +0000 UTC m=+36.486686624 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls") pod "image-registry-59b8cd454d-pxwjg" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7") : secret "image-registry-tls" not found
Jul 21 20:40:08.776717 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-bound-sa-token\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.776963 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776745    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-trusted-ca\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.776963 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776777    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.776963 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776801    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prkg5\" (UniqueName: \"kubernetes.io/projected/820520d0-8efa-4215-bbe9-30746248ce48-kube-api-access-prkg5\") pod \"ingress-canary-p5j5p\" (UID: \"820520d0-8efa-4215-bbe9-30746248ce48\") " pod="openshift-ingress-canary/ingress-canary-p5j5p"
Jul 21 20:40:08.776963 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776834    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/820520d0-8efa-4215-bbe9-30746248ce48-cert\") pod \"ingress-canary-p5j5p\" (UID: \"820520d0-8efa-4215-bbe9-30746248ce48\") " pod="openshift-ingress-canary/ingress-canary-p5j5p"
Jul 21 20:40:08.776963 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776870    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-installation-pull-secrets\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.776963 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776904    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbst2\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-kube-api-access-sbst2\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.776963 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.776939    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bnbw\" (UniqueName: \"kubernetes.io/projected/2879e13a-d9ac-4b68-8353-c8f2d717423f-kube-api-access-7bnbw\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.777321 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.777041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2879e13a-d9ac-4b68-8353-c8f2d717423f-config-volume\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.777321 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.777084    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2879e13a-d9ac-4b68-8353-c8f2d717423f-tmp-dir\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.777321 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.777146    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ff55187b-7adc-4356-84cb-8c3830f02ed7-ca-trust-extracted\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.777462 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.777341    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-certificates\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.777517 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.777494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ff55187b-7adc-4356-84cb-8c3830f02ed7-ca-trust-extracted\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.778044 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.778018    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-trusted-ca\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.781373 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.781351    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-installation-pull-secrets\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.781373 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.781368    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-image-registry-private-configuration\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.789225 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.789199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbst2\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-kube-api-access-sbst2\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.789786 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.789766    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-bound-sa-token\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:08.878101 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.877973    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.878101 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.878029    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prkg5\" (UniqueName: \"kubernetes.io/projected/820520d0-8efa-4215-bbe9-30746248ce48-kube-api-access-prkg5\") pod \"ingress-canary-p5j5p\" (UID: \"820520d0-8efa-4215-bbe9-30746248ce48\") " pod="openshift-ingress-canary/ingress-canary-p5j5p"
Jul 21 20:40:08.878101 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.878059    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/820520d0-8efa-4215-bbe9-30746248ce48-cert\") pod \"ingress-canary-p5j5p\" (UID: \"820520d0-8efa-4215-bbe9-30746248ce48\") " pod="openshift-ingress-canary/ingress-canary-p5j5p"
Jul 21 20:40:08.878355 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:08.878132    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 20:40:08.878355 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:08.878196    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls podName:2879e13a-d9ac-4b68-8353-c8f2d717423f nodeName:}" failed. No retries permitted until 2026-07-21 20:40:09.378178706 +0000 UTC m=+36.588207109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls") pod "dns-default-7qx6q" (UID: "2879e13a-d9ac-4b68-8353-c8f2d717423f") : secret "dns-default-metrics-tls" not found
Jul 21 20:40:08.878355 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.878215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bnbw\" (UniqueName: \"kubernetes.io/projected/2879e13a-d9ac-4b68-8353-c8f2d717423f-kube-api-access-7bnbw\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.878355 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.878259    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2879e13a-d9ac-4b68-8353-c8f2d717423f-config-volume\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.878355 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.878274    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2879e13a-d9ac-4b68-8353-c8f2d717423f-tmp-dir\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.878595 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.878578    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2879e13a-d9ac-4b68-8353-c8f2d717423f-tmp-dir\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.878791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.878774    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2879e13a-d9ac-4b68-8353-c8f2d717423f-config-volume\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.880307 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.880287    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/820520d0-8efa-4215-bbe9-30746248ce48-cert\") pod \"ingress-canary-p5j5p\" (UID: \"820520d0-8efa-4215-bbe9-30746248ce48\") " pod="openshift-ingress-canary/ingress-canary-p5j5p"
Jul 21 20:40:08.888134 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.888109    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prkg5\" (UniqueName: \"kubernetes.io/projected/820520d0-8efa-4215-bbe9-30746248ce48-kube-api-access-prkg5\") pod \"ingress-canary-p5j5p\" (UID: \"820520d0-8efa-4215-bbe9-30746248ce48\") " pod="openshift-ingress-canary/ingress-canary-p5j5p"
Jul 21 20:40:08.888254 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.888238    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bnbw\" (UniqueName: \"kubernetes.io/projected/2879e13a-d9ac-4b68-8353-c8f2d717423f-kube-api-access-7bnbw\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:08.942780 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:08.942752    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p5j5p"
Jul 21 20:40:09.092191 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:09.092165    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p5j5p"]
Jul 21 20:40:09.095378 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:40:09.095342    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod820520d0_8efa_4215_bbe9_30746248ce48.slice/crio-966d5c4eda19fee2a5456cba3dd977c50bb0984b2f3c1de72c526644aec5ca3e WatchSource:0}: Error finding container 966d5c4eda19fee2a5456cba3dd977c50bb0984b2f3c1de72c526644aec5ca3e: Status 404 returned error can't find the container with id 966d5c4eda19fee2a5456cba3dd977c50bb0984b2f3c1de72c526644aec5ca3e
Jul 21 20:40:09.282038 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:09.282007    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:09.282198 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:09.282120    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 20:40:09.282198 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:09.282132    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b8cd454d-pxwjg: secret "image-registry-tls" not found
Jul 21 20:40:09.282198 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:09.282184    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls podName:ff55187b-7adc-4356-84cb-8c3830f02ed7 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:10.282169915 +0000 UTC m=+37.492198313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls") pod "image-registry-59b8cd454d-pxwjg" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7") : secret "image-registry-tls" not found
Jul 21 20:40:09.383436 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:09.383398    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:09.383841 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:09.383534    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 20:40:09.383841 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:09.383593    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls podName:2879e13a-d9ac-4b68-8353-c8f2d717423f nodeName:}" failed. No retries permitted until 2026-07-21 20:40:10.383576807 +0000 UTC m=+37.593605228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls") pod "dns-default-7qx6q" (UID: "2879e13a-d9ac-4b68-8353-c8f2d717423f") : secret "dns-default-metrics-tls" not found
Jul 21 20:40:09.478994 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:09.478962    2567 generic.go:356] "Generic (PLEG): container finished" podID="d3004918-748e-45ab-85e8-7eefa1d359b3" containerID="a252c6ef81faa97e07196c2b4e3b0ba78c4625cf0fb919c37b7eecf815221e2b" exitCode=0
Jul 21 20:40:09.479185 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:09.479052    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" event={"ID":"d3004918-748e-45ab-85e8-7eefa1d359b3","Type":"ContainerDied","Data":"a252c6ef81faa97e07196c2b4e3b0ba78c4625cf0fb919c37b7eecf815221e2b"}
Jul 21 20:40:09.480399 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:09.480194    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p5j5p" event={"ID":"820520d0-8efa-4215-bbe9-30746248ce48","Type":"ContainerStarted","Data":"966d5c4eda19fee2a5456cba3dd977c50bb0984b2f3c1de72c526644aec5ca3e"}
Jul 21 20:40:10.290350 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.290108    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:10.290538 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:10.290281    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 20:40:10.290538 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:10.290410    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b8cd454d-pxwjg: secret "image-registry-tls" not found
Jul 21 20:40:10.290538 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:10.290477    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls podName:ff55187b-7adc-4356-84cb-8c3830f02ed7 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:12.290456472 +0000 UTC m=+39.500484892 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls") pod "image-registry-59b8cd454d-pxwjg" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7") : secret "image-registry-tls" not found
Jul 21 20:40:10.305126 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.305090    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:10.305282 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.305244    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:10.305418 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.305396    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:40:10.308116 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.308095    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 20:40:10.308236 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.308130    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 20:40:10.308288 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.308261    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 20:40:10.309603 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.309531    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 20:40:10.309603 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.309575    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdqrl\""
Jul 21 20:40:10.309603 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.309581    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jpthn\""
Jul 21 20:40:10.391539 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.391502    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:10.391905 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:10.391658    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 20:40:10.391905 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:10.391722    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls podName:2879e13a-d9ac-4b68-8353-c8f2d717423f nodeName:}" failed. No retries permitted until 2026-07-21 20:40:12.391707897 +0000 UTC m=+39.601736296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls") pod "dns-default-7qx6q" (UID: "2879e13a-d9ac-4b68-8353-c8f2d717423f") : secret "dns-default-metrics-tls" not found
Jul 21 20:40:10.487774 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.487734    2567 generic.go:356] "Generic (PLEG): container finished" podID="d3004918-748e-45ab-85e8-7eefa1d359b3" containerID="8e2cc2df8cce4a0f79999a2babb9cc13a4b68686a3886e0e065ce3eea4b1c1dd" exitCode=0
Jul 21 20:40:10.487945 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:10.487786    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" event={"ID":"d3004918-748e-45ab-85e8-7eefa1d359b3","Type":"ContainerDied","Data":"8e2cc2df8cce4a0f79999a2babb9cc13a4b68686a3886e0e065ce3eea4b1c1dd"}
Jul 21 20:40:11.493975 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:11.493939    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" event={"ID":"d3004918-748e-45ab-85e8-7eefa1d359b3","Type":"ContainerStarted","Data":"695b502b1360bf0a0f9da4a57accd4056367615b7895ddf94290cc7330a0691d"}
Jul 21 20:40:11.521166 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:11.521041    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wc6xm" podStartSLOduration=8.555957351 podStartE2EDuration="38.521019851s" totalImagesPullingTime="29.9650625s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:11.519849612 +0000 UTC m=+38.729878053" watchObservedRunningTime="2026-07-21 20:40:11.521019851 +0000 UTC m=+38.731048276"
Jul 21 20:40:11.668400 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:11.668358    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:11.767356 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:11.767270    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:12.306733 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:12.306692    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:12.306938 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:12.306922    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 20:40:12.306979 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:12.306941    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b8cd454d-pxwjg: secret "image-registry-tls" not found
Jul 21 20:40:12.307046 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:12.307009    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls podName:ff55187b-7adc-4356-84cb-8c3830f02ed7 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:16.306979141 +0000 UTC m=+43.517007540 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls") pod "image-registry-59b8cd454d-pxwjg" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7") : secret "image-registry-tls" not found
Jul 21 20:40:12.369144 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:12.369062    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:12.407873 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:12.407845    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:12.408029 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:12.408012    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 20:40:12.408086 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:12.408076    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls podName:2879e13a-d9ac-4b68-8353-c8f2d717423f nodeName:}" failed. No retries permitted until 2026-07-21 20:40:16.408057781 +0000 UTC m=+43.618086180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls") pod "dns-default-7qx6q" (UID: "2879e13a-d9ac-4b68-8353-c8f2d717423f") : secret "dns-default-metrics-tls" not found
Jul 21 20:40:12.470624 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:12.470596    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:12.497306 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:12.497270    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p5j5p" event={"ID":"820520d0-8efa-4215-bbe9-30746248ce48","Type":"ContainerStarted","Data":"c6672dff131631b605a660a94b7f4159c84ce17175d05d6fe0f246b4d6e2fc76"}
Jul 21 20:40:12.514650 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:12.514603    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p5j5p" podStartSLOduration=1.5441778130000001 podStartE2EDuration="4.51458877s" totalImagesPullingTime="2.970410957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:12.513123458 +0000 UTC m=+39.723151900" watchObservedRunningTime="2026-07-21 20:40:12.51458877 +0000 UTC m=+39.724617235"
Jul 21 20:40:14.039461 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:14.039426    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:14.146827 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:14.146791    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:14.241684 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:14.241651    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:16.337095 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:16.337061    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:16.337447 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:16.337212    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 20:40:16.337447 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:16.337231    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b8cd454d-pxwjg: secret "image-registry-tls" not found
Jul 21 20:40:16.337447 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:16.337287    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls podName:ff55187b-7adc-4356-84cb-8c3830f02ed7 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:24.337271033 +0000 UTC m=+51.547299431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls") pod "image-registry-59b8cd454d-pxwjg" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7") : secret "image-registry-tls" not found
Jul 21 20:40:16.437649 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:16.437616    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:16.437791 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:16.437770    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 20:40:16.437848 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:16.437839    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls podName:2879e13a-d9ac-4b68-8353-c8f2d717423f nodeName:}" failed. No retries permitted until 2026-07-21 20:40:24.437824119 +0000 UTC m=+51.647852518 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls") pod "dns-default-7qx6q" (UID: "2879e13a-d9ac-4b68-8353-c8f2d717423f") : secret "dns-default-metrics-tls" not found
Jul 21 20:40:17.140994 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:17.140956    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:17.471394 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:17.471362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:23.935117 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:23.935082    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:24.012449 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:24.012421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:24.396241 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:24.396160    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:24.396381 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:24.396315    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 20:40:24.396381 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:24.396335    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b8cd454d-pxwjg: secret "image-registry-tls" not found
Jul 21 20:40:24.396451 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:24.396398    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls podName:ff55187b-7adc-4356-84cb-8c3830f02ed7 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:40.396376127 +0000 UTC m=+67.606404526 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls") pod "image-registry-59b8cd454d-pxwjg" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7") : secret "image-registry-tls" not found
Jul 21 20:40:24.497423 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:24.497385    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:24.497550 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:24.497506    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 20:40:24.497587 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:24.497556    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls podName:2879e13a-d9ac-4b68-8353-c8f2d717423f nodeName:}" failed. No retries permitted until 2026-07-21 20:40:40.497543182 +0000 UTC m=+67.707571584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls") pod "dns-default-7qx6q" (UID: "2879e13a-d9ac-4b68-8353-c8f2d717423f") : secret "dns-default-metrics-tls" not found
Jul 21 20:40:24.910495 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:24.910458    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:24.989352 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:24.989321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:25.967005 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:25.966961    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:26.057079 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:26.057038    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:28.320450 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.320416    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lk8hn"]
Jul 21 20:40:28.322666 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.322650    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.324934 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.324908    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 21 20:40:28.325258 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.325231    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5503f039-d91f-464c-8731-8dbbecc97c04-snapshots\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.325410 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.325318    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 20:40:28.325410 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.325328    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5503f039-d91f-464c-8731-8dbbecc97c04-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.325410 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.325365    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5503f039-d91f-464c-8731-8dbbecc97c04-service-ca-bundle\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.325410 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.325388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5503f039-d91f-464c-8731-8dbbecc97c04-tmp\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.325410 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.325318    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 20:40:28.325410 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.325412    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5503f039-d91f-464c-8731-8dbbecc97c04-serving-cert\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.325657 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.325442    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8lsj\" (UniqueName: \"kubernetes.io/projected/5503f039-d91f-464c-8731-8dbbecc97c04-kube-api-access-p8lsj\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.326305 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.326288    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-57mwx\""
Jul 21 20:40:28.326395 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.326306    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 21 20:40:28.331259 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.331239    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 21 20:40:28.334246 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.334225    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lk8hn"]
Jul 21 20:40:28.421767 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.421735    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"]
Jul 21 20:40:28.423677 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.423663    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.426207 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426172    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5503f039-d91f-464c-8731-8dbbecc97c04-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.426322 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426212    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.426322 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426250    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5503f039-d91f-464c-8731-8dbbecc97c04-service-ca-bundle\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.426322 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5503f039-d91f-464c-8731-8dbbecc97c04-tmp\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.426480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5503f039-d91f-464c-8731-8dbbecc97c04-serving-cert\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.426480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426366    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8lsj\" (UniqueName: \"kubernetes.io/projected/5503f039-d91f-464c-8731-8dbbecc97c04-kube-api-access-p8lsj\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.426480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426467    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 21 20:40:28.426632 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426467    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9h68\" (UniqueName: \"kubernetes.io/projected/2bf8996a-81f7-490c-acc0-a201ff2a902e-kube-api-access-m9h68\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.426632 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426549    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5503f039-d91f-464c-8731-8dbbecc97c04-snapshots\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.426632 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426571    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2bf8996a-81f7-490c-acc0-a201ff2a902e-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.426759 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.426629    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5503f039-d91f-464c-8731-8dbbecc97c04-tmp\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.427145 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.427126    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5503f039-d91f-464c-8731-8dbbecc97c04-snapshots\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.427193 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.427134    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5503f039-d91f-464c-8731-8dbbecc97c04-service-ca-bundle\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.427550 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.427503    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5503f039-d91f-464c-8731-8dbbecc97c04-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.427664 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.427600    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lsrgl\""
Jul 21 20:40:28.427664 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.427651    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 20:40:28.427664 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.427607    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 20:40:28.427790 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.427672    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 21 20:40:28.430255 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.430232    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5503f039-d91f-464c-8731-8dbbecc97c04-serving-cert\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.439126 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.439102    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"]
Jul 21 20:40:28.442326 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.442300    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8lsj\" (UniqueName: \"kubernetes.io/projected/5503f039-d91f-464c-8731-8dbbecc97c04-kube-api-access-p8lsj\") pod \"insights-operator-555d9d7885-lk8hn\" (UID: \"5503f039-d91f-464c-8731-8dbbecc97c04\") " pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.527676 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.527639    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2bf8996a-81f7-490c-acc0-a201ff2a902e-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.527833 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.527715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.527833 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.527753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9h68\" (UniqueName: \"kubernetes.io/projected/2bf8996a-81f7-490c-acc0-a201ff2a902e-kube-api-access-m9h68\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.527909 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:28.527840    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:28.527942 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:28.527919    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls podName:2bf8996a-81f7-490c-acc0-a201ff2a902e nodeName:}" failed. No retries permitted until 2026-07-21 20:40:29.027900529 +0000 UTC m=+56.237928932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-plspl" (UID: "2bf8996a-81f7-490c-acc0-a201ff2a902e") : secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:28.528366 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.528350    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2bf8996a-81f7-490c-acc0-a201ff2a902e-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.544179 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.544151    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9h68\" (UniqueName: \"kubernetes.io/projected/2bf8996a-81f7-490c-acc0-a201ff2a902e-kube-api-access-m9h68\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:28.609912 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.609827    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:28.632241 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.632210    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-lk8hn"
Jul 21 20:40:28.686944 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.686901    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:28.762401 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.762345    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-lk8hn"]
Jul 21 20:40:28.766149 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:40:28.766125    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5503f039_d91f_464c_8731_8dbbecc97c04.slice/crio-fef585e821ba40b4c4371a36e4853a71ef8d606007170415a6b866a31fa43158 WatchSource:0}: Error finding container fef585e821ba40b4c4371a36e4853a71ef8d606007170415a6b866a31fa43158: Status 404 returned error can't find the container with id fef585e821ba40b4c4371a36e4853a71ef8d606007170415a6b866a31fa43158
Jul 21 20:40:28.950750 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:28.950714    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:29.030905 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:29.030859    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:29.031089 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:29.031018    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:29.031089 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:29.031079    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls podName:2bf8996a-81f7-490c-acc0-a201ff2a902e nodeName:}" failed. No retries permitted until 2026-07-21 20:40:30.031063326 +0000 UTC m=+57.241091728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-plspl" (UID: "2bf8996a-81f7-490c-acc0-a201ff2a902e") : secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:29.538054 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:29.538016    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lk8hn" event={"ID":"5503f039-d91f-464c-8731-8dbbecc97c04","Type":"ContainerStarted","Data":"fef585e821ba40b4c4371a36e4853a71ef8d606007170415a6b866a31fa43158"}
Jul 21 20:40:30.039542 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.039503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:30.039736 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:30.039683    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:30.039804 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:30.039769    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls podName:2bf8996a-81f7-490c-acc0-a201ff2a902e nodeName:}" failed. No retries permitted until 2026-07-21 20:40:32.039748814 +0000 UTC m=+59.249777220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-plspl" (UID: "2bf8996a-81f7-490c-acc0-a201ff2a902e") : secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:30.154296 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.154259    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"]
Jul 21 20:40:30.156766 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.156745    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.159958 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.159924    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 21 20:40:30.160892 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.160869    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hnzwz\""
Jul 21 20:40:30.161013 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.160888    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 21 20:40:30.161013 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.160908    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 21 20:40:30.161013 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.160889    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 21 20:40:30.167317 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.167294    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"]
Jul 21 20:40:30.241230 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.241193    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79c9b2be-4e0d-4273-b125-1144be8c52de-config\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.241386 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.241306    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkqss\" (UniqueName: \"kubernetes.io/projected/79c9b2be-4e0d-4273-b125-1144be8c52de-kube-api-access-tkqss\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.241386 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.241378    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79c9b2be-4e0d-4273-b125-1144be8c52de-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.342495 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.342416    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79c9b2be-4e0d-4273-b125-1144be8c52de-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.342495 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.342449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79c9b2be-4e0d-4273-b125-1144be8c52de-config\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.342659 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.342509    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkqss\" (UniqueName: \"kubernetes.io/projected/79c9b2be-4e0d-4273-b125-1144be8c52de-kube-api-access-tkqss\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.343100 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.343079    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79c9b2be-4e0d-4273-b125-1144be8c52de-config\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.344584 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.344567    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79c9b2be-4e0d-4273-b125-1144be8c52de-serving-cert\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.351309 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.351291    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkqss\" (UniqueName: \"kubernetes.io/projected/79c9b2be-4e0d-4273-b125-1144be8c52de-kube-api-access-tkqss\") pod \"service-ca-operator-5bcbc94c78-nkkh5\" (UID: \"79c9b2be-4e0d-4273-b125-1144be8c52de\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.467253 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.467219    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"
Jul 21 20:40:30.611114 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.611046    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5"]
Jul 21 20:40:30.996055 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:30.996023    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:31.050299 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:40:31.050263    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c9b2be_4e0d_4273_b125_1144be8c52de.slice/crio-9cfd119f40d64d36ca79358988f4f0ca35c916b6a3dad1ad2ba5615329c43880 WatchSource:0}: Error finding container 9cfd119f40d64d36ca79358988f4f0ca35c916b6a3dad1ad2ba5615329c43880: Status 404 returned error can't find the container with id 9cfd119f40d64d36ca79358988f4f0ca35c916b6a3dad1ad2ba5615329c43880
Jul 21 20:40:31.096559 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:31.096529    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:31.545278 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:31.545239    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5" event={"ID":"79c9b2be-4e0d-4273-b125-1144be8c52de","Type":"ContainerStarted","Data":"9cfd119f40d64d36ca79358988f4f0ca35c916b6a3dad1ad2ba5615329c43880"}
Jul 21 20:40:31.547040 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:31.547011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lk8hn" event={"ID":"5503f039-d91f-464c-8731-8dbbecc97c04","Type":"ContainerStarted","Data":"a70efb1f158d9029a1e2b84a83e66273914dd27d43fb9d7b823933dd9f1a1aac"}
Jul 21 20:40:31.581028 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:31.580954    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-lk8hn" podStartSLOduration=1.254485106 podStartE2EDuration="3.580932452s" totalImagesPullingTime="2.326447346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:31.579830312 +0000 UTC m=+58.789858764" watchObservedRunningTime="2026-07-21 20:40:31.580932452 +0000 UTC m=+58.790960871"
Jul 21 20:40:32.061125 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.061084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:32.061495 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:32.061234    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:32.061495 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:32.061300    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls podName:2bf8996a-81f7-490c-acc0-a201ff2a902e nodeName:}" failed. No retries permitted until 2026-07-21 20:40:36.061284871 +0000 UTC m=+63.271313271 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-plspl" (UID: "2bf8996a-81f7-490c-acc0-a201ff2a902e") : secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:32.652821 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.652642    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bvwwn"]
Jul 21 20:40:32.658489 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.658452    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.662317 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:32.662289    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-sa-dockercfg-qwsp9\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qwsp9\"" type="*v1.Secret"
Jul 21 20:40:32.662836 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:32.662791    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"insights-runtime-extractor-bvwwn\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" podUID="37d8148e-f700-4a83-924f-a60ce06386e9" pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.662836 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:32.662804    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"insights-runtime-extractor-tls\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-insights\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" type="*v1.Secret"
Jul 21 20:40:32.665064 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.665041    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 20:40:32.679837 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.679815    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bvwwn"]
Jul 21 20:40:32.766215 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.766180    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:32.766362 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.766332    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37d8148e-f700-4a83-924f-a60ce06386e9-crio-socket\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.766418 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.766407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37d8148e-f700-4a83-924f-a60ce06386e9-data-volume\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.766454 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.766438    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.766501 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.766479    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.766541 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.766510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrbvw\" (UniqueName: \"kubernetes.io/projected/37d8148e-f700-4a83-924f-a60ce06386e9-kube-api-access-hrbvw\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.866929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.866891    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37d8148e-f700-4a83-924f-a60ce06386e9-data-volume\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.867143 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.866939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.867143 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.867027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.867143 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.867064    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrbvw\" (UniqueName: \"kubernetes.io/projected/37d8148e-f700-4a83-924f-a60ce06386e9-kube-api-access-hrbvw\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.867143 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.867099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37d8148e-f700-4a83-924f-a60ce06386e9-crio-socket\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.867366 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.867206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/37d8148e-f700-4a83-924f-a60ce06386e9-crio-socket\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.867366 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:32.867285    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 20:40:32.867366 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.867319    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/37d8148e-f700-4a83-924f-a60ce06386e9-data-volume\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.867366 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:32.867348    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:33.367329994 +0000 UTC m=+60.577358414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : configmap "kube-rbac-proxy" not found
Jul 21 20:40:32.891409 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.891382    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrbvw\" (UniqueName: \"kubernetes.io/projected/37d8148e-f700-4a83-924f-a60ce06386e9-kube-api-access-hrbvw\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:32.926756 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:32.926721    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:33.022054 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.022015    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:33.375371 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.375267    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:33.375788 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:33.375409    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 20:40:33.375788 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:33.375487    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:34.375466089 +0000 UTC m=+61.585494508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : configmap "kube-rbac-proxy" not found
Jul 21 20:40:33.553848 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.553817    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5" event={"ID":"79c9b2be-4e0d-4273-b125-1144be8c52de","Type":"ContainerStarted","Data":"55fa7d6243969f306bf9a39adaeeaee7e1b556dfd185e1f679d253bdbd596b47"}
Jul 21 20:40:33.571701 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.571654    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5" podStartSLOduration=1.129894232 podStartE2EDuration="3.571633907s" totalImagesPullingTime="2.441739675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:33.571141795 +0000 UTC m=+60.781170218" watchObservedRunningTime="2026-07-21 20:40:33.571633907 +0000 UTC m=+60.781662328"
Jul 21 20:40:33.753198 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.753165    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42"]
Jul 21 20:40:33.776150 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.776118    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42"]
Jul 21 20:40:33.776287 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.776220    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42"
Jul 21 20:40:33.778982 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.778959    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 21 20:40:33.778982 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.778970    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 21 20:40:33.779259 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.779246    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tqcl8\""
Jul 21 20:40:33.784825 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.784804    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qwsp9\""
Jul 21 20:40:33.801418 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.801386    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:33.868331 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:33.868297    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: failed to sync secret cache: timed out waiting for the condition
Jul 21 20:40:33.868496 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:33.868373    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:34.368357497 +0000 UTC m=+61.578385897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : failed to sync secret cache: timed out waiting for the condition
Jul 21 20:40:33.875792 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.875762    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:33.880039 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.880013    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkkkm\" (UniqueName: \"kubernetes.io/projected/772a7ff6-26b0-445a-a319-5eb76fd9b180-kube-api-access-nkkkm\") pod \"migrator-7cc9c7ffd7-vrg42\" (UID: \"772a7ff6-26b0-445a-a319-5eb76fd9b180\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42"
Jul 21 20:40:33.981441 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.981397    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkkkm\" (UniqueName: \"kubernetes.io/projected/772a7ff6-26b0-445a-a319-5eb76fd9b180-kube-api-access-nkkkm\") pod \"migrator-7cc9c7ffd7-vrg42\" (UID: \"772a7ff6-26b0-445a-a319-5eb76fd9b180\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42"
Jul 21 20:40:33.990410 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:33.990381    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkkkm\" (UniqueName: \"kubernetes.io/projected/772a7ff6-26b0-445a-a319-5eb76fd9b180-kube-api-access-nkkkm\") pod \"migrator-7cc9c7ffd7-vrg42\" (UID: \"772a7ff6-26b0-445a-a319-5eb76fd9b180\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42"
Jul 21 20:40:34.064440 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.064366    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n2h4z_a56b897c-e624-4d27-a9bb-8e36b5121ce2/dns-node-resolver/0.log"
Jul 21 20:40:34.084196 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.084162    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42"
Jul 21 20:40:34.214780 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.214753    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 20:40:34.225509 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.225475    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42"]
Jul 21 20:40:34.229838 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:40:34.229810    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772a7ff6_26b0_445a_a319_5eb76fd9b180.slice/crio-be08fd0ca0919c4e4f3021faffad7ddbd20464ba1dc800d7bf64296ddceafec2 WatchSource:0}: Error finding container be08fd0ca0919c4e4f3021faffad7ddbd20464ba1dc800d7bf64296ddceafec2: Status 404 returned error can't find the container with id be08fd0ca0919c4e4f3021faffad7ddbd20464ba1dc800d7bf64296ddceafec2
Jul 21 20:40:34.384308 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.384232    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:34.384666 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.384357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:34.384666 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:34.384397    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:34.384666 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:34.384425    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 20:40:34.384666 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:34.384467    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:35.38444923 +0000 UTC m=+62.594477659 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:34.384666 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:34.384536    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:36.384514442 +0000 UTC m=+63.594542844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : configmap "kube-rbac-proxy" not found
Jul 21 20:40:34.481535 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.481508    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pv66t"
Jul 21 20:40:34.557265 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.557235    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42" event={"ID":"772a7ff6-26b0-445a-a319-5eb76fd9b180","Type":"ContainerStarted","Data":"be08fd0ca0919c4e4f3021faffad7ddbd20464ba1dc800d7bf64296ddceafec2"}
Jul 21 20:40:34.656822 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.656747    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:34.725017 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:34.724941    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:35.264387 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:35.264358    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lcbgb_33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb/node-ca/0.log"
Jul 21 20:40:35.393403 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:35.393366    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:35.393870 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:35.393547    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:35.393870 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:35.393624    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:37.393606991 +0000 UTC m=+64.603635408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:35.561450 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:35.561418    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42" event={"ID":"772a7ff6-26b0-445a-a319-5eb76fd9b180","Type":"ContainerStarted","Data":"f45e7073f16a796dcac0e771da27f8b0ff212cdccaf03e234dbbde5e8f3a36f3"}
Jul 21 20:40:35.648140 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:35.648097    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:35.749895 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:35.749861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:35.949489 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:35.949453    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:36.048678 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.048644    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:36.064830 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.064803    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p5j5p_820520d0-8efa-4215-bbe9-30746248ce48/serve-healthcheck-canary/0.log"
Jul 21 20:40:36.097241 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.097211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:36.097380 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:36.097330    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:36.097434 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:36.097386    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls podName:2bf8996a-81f7-490c-acc0-a201ff2a902e nodeName:}" failed. No retries permitted until 2026-07-21 20:40:44.097369473 +0000 UTC m=+71.307397887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-plspl" (UID: "2bf8996a-81f7-490c-acc0-a201ff2a902e") : secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:36.149297 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.149265    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:36.353370 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.353295    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:36.400552 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.400525    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:36.400879 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:36.400622    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 20:40:36.400879 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:36.400697    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:40.400683031 +0000 UTC m=+67.610711430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : configmap "kube-rbac-proxy" not found
Jul 21 20:40:36.458381 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.458354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:36.566362 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.566326    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42" event={"ID":"772a7ff6-26b0-445a-a319-5eb76fd9b180","Type":"ContainerStarted","Data":"d927273e50267532db39e1c3f77a5c0c6d1d92bba9e7d360500d134575554227"}
Jul 21 20:40:36.584220 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.584176    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-vrg42" podStartSLOduration=2.320987915 podStartE2EDuration="3.584161179s" totalImagesPullingTime="1.263173264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:36.583702783 +0000 UTC m=+63.793731214" watchObservedRunningTime="2026-07-21 20:40:36.584161179 +0000 UTC m=+63.794189600"
Jul 21 20:40:36.683096 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.683063    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:36.783336 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:36.783301    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:37.409390 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:37.409353    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:37.409763 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:37.409479    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:37.409763 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:37.409532    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:41.409518184 +0000 UTC m=+68.619546586 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:37.591021 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:37.590965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:37.653398 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:37.653356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:37.981048 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:37.981011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:38.045553 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:38.045514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:38.777219 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:38.777188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:39.024269 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.024237    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:39.024451 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.024303    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:40:39.027030 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.027012    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 20:40:39.027960 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.027917    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 20:40:39.034665 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:39.034646    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 21 20:40:39.034728 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:39.034703    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs podName:cf884bc1-53a5-47ab-98cd-3959ded9e6e7 nodeName:}" failed. No retries permitted until 2026-07-21 20:41:43.03468792 +0000 UTC m=+130.244716333 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs") pod "network-metrics-daemon-tms9z" (UID: "cf884bc1-53a5-47ab-98cd-3959ded9e6e7") : secret "metrics-daemon-secret" not found
Jul 21 20:40:39.036776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.036756    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf604c6d-39e1-43a4-8cfc-9f06bf3cd729-original-pull-secret\") pod \"global-pull-secret-syncer-pdlgq\" (UID: \"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729\") " pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:39.117579 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.117545    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pdlgq"
Jul 21 20:40:39.124789 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.124768    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:39.127462 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.127434    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 20:40:39.138225 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.138201    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 20:40:39.148602 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.148565    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hffth\" (UniqueName: \"kubernetes.io/projected/6e320cc6-4317-4840-90a4-2d07ff82d19c-kube-api-access-hffth\") pod \"network-check-target-8cksp\" (UID: \"6e320cc6-4317-4840-90a4-2d07ff82d19c\") " pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:39.251502 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.251467    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pdlgq"]
Jul 21 20:40:39.254428 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:40:39.254405    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf604c6d_39e1_43a4_8cfc_9f06bf3cd729.slice/crio-a855fd6263527361064fb675c4dbdcd7d2ba462d0c6e0acf9960894e78b9350d WatchSource:0}: Error finding container a855fd6263527361064fb675c4dbdcd7d2ba462d0c6e0acf9960894e78b9350d: Status 404 returned error can't find the container with id a855fd6263527361064fb675c4dbdcd7d2ba462d0c6e0acf9960894e78b9350d
Jul 21 20:40:39.426707 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.426678    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdqrl\""
Jul 21 20:40:39.435044 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.435027    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:39.568240 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.568207    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8cksp"]
Jul 21 20:40:39.572755 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:40:39.572727    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e320cc6_4317_4840_90a4_2d07ff82d19c.slice/crio-2ce9bfe77f4bc9fcdffe03cfa9b131cf036df45e6679135f3542ecbbe284d3fd WatchSource:0}: Error finding container 2ce9bfe77f4bc9fcdffe03cfa9b131cf036df45e6679135f3542ecbbe284d3fd: Status 404 returned error can't find the container with id 2ce9bfe77f4bc9fcdffe03cfa9b131cf036df45e6679135f3542ecbbe284d3fd
Jul 21 20:40:39.576158 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:39.576129    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pdlgq" event={"ID":"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729","Type":"ContainerStarted","Data":"a855fd6263527361064fb675c4dbdcd7d2ba462d0c6e0acf9960894e78b9350d"}
Jul 21 20:40:40.436484 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:40.436444    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:40.437045 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:40.436497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:40:40.437045 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:40.436569    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 20:40:40.437045 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:40.436616    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 20:40:40.437045 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:40.436629    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-59b8cd454d-pxwjg: secret "image-registry-tls" not found
Jul 21 20:40:40.437045 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:40.436639    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:48.436617579 +0000 UTC m=+75.646645994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : configmap "kube-rbac-proxy" not found
Jul 21 20:40:40.437045 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:40.436660    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls podName:ff55187b-7adc-4356-84cb-8c3830f02ed7 nodeName:}" failed. No retries permitted until 2026-07-21 20:41:12.43665021 +0000 UTC m=+99.646678626 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls") pod "image-registry-59b8cd454d-pxwjg" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7") : secret "image-registry-tls" not found
Jul 21 20:40:40.537944 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:40.537904    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:40:40.538116 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:40.538047    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 20:40:40.538192 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:40.538133    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls podName:2879e13a-d9ac-4b68-8353-c8f2d717423f nodeName:}" failed. No retries permitted until 2026-07-21 20:41:12.538111662 +0000 UTC m=+99.748140075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls") pod "dns-default-7qx6q" (UID: "2879e13a-d9ac-4b68-8353-c8f2d717423f") : secret "dns-default-metrics-tls" not found
Jul 21 20:40:40.581509 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:40.581468    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8cksp" event={"ID":"6e320cc6-4317-4840-90a4-2d07ff82d19c","Type":"ContainerStarted","Data":"2ce9bfe77f4bc9fcdffe03cfa9b131cf036df45e6679135f3542ecbbe284d3fd"}
Jul 21 20:40:40.999260 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:40.999208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:41.151462 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:41.151331    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:41.445825 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:41.445785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:41.446331 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:41.446005    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:41.446331 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:41.446085    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:40:49.446064006 +0000 UTC m=+76.656092428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:43.640763 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:43.640723    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:43.808567 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:43.808516    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:44.048346 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.048296    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:44.171238 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.171203    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:40:44.171366 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:44.171349    2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:44.171429 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:44.171409    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls podName:2bf8996a-81f7-490c-acc0-a201ff2a902e nodeName:}" failed. No retries permitted until 2026-07-21 20:41:00.171393678 +0000 UTC m=+87.381422077 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-plspl" (UID: "2bf8996a-81f7-490c-acc0-a201ff2a902e") : secret "cluster-monitoring-operator-tls" not found
Jul 21 20:40:44.193416 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.193382    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:44.285644 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.285605    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:44.386416 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.386335    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:44.490521 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.490491    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:44.597120 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.597088    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pdlgq" event={"ID":"cf604c6d-39e1-43a4-8cfc-9f06bf3cd729","Type":"ContainerStarted","Data":"a07a7bfc1d4045f01224aa602c40b752cf4de21e08710e57a3c1b0e246190a4f"}
Jul 21 20:40:44.598458 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.598434    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8cksp" event={"ID":"6e320cc6-4317-4840-90a4-2d07ff82d19c","Type":"ContainerStarted","Data":"f4c6882d0f661b7fcc75e59d599c71db134c30798e500773ba2ba4eb228402c0"}
Jul 21 20:40:44.598590 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.598573    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:40:44.613501 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.613457    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pdlgq" podStartSLOduration=66.78367691 podStartE2EDuration="1m11.613443164s" totalImagesPullingTime="4.829766254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:44.612677228 +0000 UTC m=+71.822705669" watchObservedRunningTime="2026-07-21 20:40:44.613443164 +0000 UTC m=+71.823471585"
Jul 21 20:40:44.629077 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:44.629037    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8cksp" podStartSLOduration=67.123161326 podStartE2EDuration="1m11.629023084s" totalImagesPullingTime="4.505861758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:40:44.62873674 +0000 UTC m=+71.838765178" watchObservedRunningTime="2026-07-21 20:40:44.629023084 +0000 UTC m=+71.839051483"
Jul 21 20:40:45.483608 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:45.483569    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:45.874721 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:45.874634    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:45.930268 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:45.930229    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:46.146427 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:46.146337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:46.286731 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:46.286696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:46.487114 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:46.487075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:47.142827 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:47.142786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:47.213670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:47.213639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:47.490835 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:47.490794    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:47.583081 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:47.583041    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:47.690902 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:47.690871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:47.893939 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:47.893854    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:48.090819 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:48.090784    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:48.290580 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:48.290545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:48.491859 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:48.491824    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:48.512546 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:48.512520    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:48.513056 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:48.513038    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/37d8148e-f700-4a83-924f-a60ce06386e9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:49.498172 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:49.498130    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:49.522023 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:49.521978    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:40:49.522136 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:49.522121    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:49.522202 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:40:49.522193    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls podName:37d8148e-f700-4a83-924f-a60ce06386e9 nodeName:}" failed. No retries permitted until 2026-07-21 20:41:05.522178749 +0000 UTC m=+92.732207148 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bvwwn" (UID: "37d8148e-f700-4a83-924f-a60ce06386e9") : secret "insights-runtime-extractor-tls" not found
Jul 21 20:40:49.754748 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:49.754668    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:49.946861 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:49.946821    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:50.117450 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:50.117347    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:50.305103 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:50.305055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:50.513180 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:50.513142    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:50.969231 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:50.969194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:51.755015 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:51.754968    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:51.895509 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:51.895464    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:52.088978 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:52.088892    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:52.289175 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:52.289138    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:52.499808 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:52.499777    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:53.493371 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:53.493332    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:54.066823 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:54.066778    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:54.124864 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:54.124829    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:54.293833 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:54.293798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:54.460578 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:54.460542    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:54.562030 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:54.561996    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:54.662049 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:54.662013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:54.762598 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:54.762508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:55.141781 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:55.141694    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:55.213191 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:55.213161    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:55.533372 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:55.533338    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:55.613195 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:55.613066    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:55.933512 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:55.933481    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:56.033204 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:56.033171    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:56.096741 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:56.096712    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:56.411636 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:56.411547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:56.754495 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:56.754462    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:40:56.809995 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:40:56.809965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:00.212238 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.212194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:41:00.214527 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.214506    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2bf8996a-81f7-490c-acc0-a201ff2a902e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-plspl\" (UID: \"2bf8996a-81f7-490c-acc0-a201ff2a902e\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:41:00.238586 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.238559    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lsrgl\""
Jul 21 20:41:00.246618 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.246600    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"
Jul 21 20:41:00.374580 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.374552    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"]
Jul 21 20:41:00.377900 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.377879    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.384204 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.383848    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 21 20:41:00.384204 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.383851    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 21 20:41:00.384763 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.384739    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 21 20:41:00.384763 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.384757    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 21 20:41:00.384921 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.384741    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 21 20:41:00.384921 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.384777    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 21 20:41:00.384921 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.384794    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 21 20:41:00.398751 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.398719    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"]
Jul 21 20:41:00.399623 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.399605    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl"]
Jul 21 20:41:00.402156 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:41:00.402132    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf8996a_81f7_490c_acc0_a201ff2a902e.slice/crio-52485247c90c1d25353b40c3b7f5dfeae3dfa11ef2c4a51b361561b1d76f10e4 WatchSource:0}: Error finding container 52485247c90c1d25353b40c3b7f5dfeae3dfa11ef2c4a51b361561b1d76f10e4: Status 404 returned error can't find the container with id 52485247c90c1d25353b40c3b7f5dfeae3dfa11ef2c4a51b361561b1d76f10e4
Jul 21 20:41:00.438798 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.438771    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:00.514512 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.514433    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsxsm\" (UniqueName: \"kubernetes.io/projected/9652b851-dcff-4565-8524-277ff0ec643a-kube-api-access-qsxsm\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.514512 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.514465    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-ca\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.514512 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.514485    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-hub\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.514512 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.514499    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9652b851-dcff-4565-8524-277ff0ec643a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.514733 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.514577    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.514733 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.514643    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.588775 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.588736    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:00.615789 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.615759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.615913 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.615813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsxsm\" (UniqueName: \"kubernetes.io/projected/9652b851-dcff-4565-8524-277ff0ec643a-kube-api-access-qsxsm\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.615913 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.615835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-ca\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.615913 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.615853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-hub\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.615913 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.615867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9652b851-dcff-4565-8524-277ff0ec643a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.615913 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.615894    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.616658 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.616636    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9652b851-dcff-4565-8524-277ff0ec643a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.618495 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.618474    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.618649 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.618626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.618715 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.618697    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-ca\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.618763 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.618743    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9652b851-dcff-4565-8524-277ff0ec643a-hub\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.625891 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.625866    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsxsm\" (UniqueName: \"kubernetes.io/projected/9652b851-dcff-4565-8524-277ff0ec643a-kube-api-access-qsxsm\") pod \"cluster-proxy-proxy-agent-84bc6dc864-cdhqk\" (UID: \"9652b851-dcff-4565-8524-277ff0ec643a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.642999 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.642964    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl" event={"ID":"2bf8996a-81f7-490c-acc0-a201ff2a902e","Type":"ContainerStarted","Data":"52485247c90c1d25353b40c3b7f5dfeae3dfa11ef2c4a51b361561b1d76f10e4"}
Jul 21 20:41:00.695333 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.695302    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:00.697114 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.697098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"
Jul 21 20:41:00.826815 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:00.826785    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk"]
Jul 21 20:41:00.829498 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:41:00.829472    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9652b851_dcff_4565_8524_277ff0ec643a.slice/crio-46a284ad4eb2bea2d22bc6973ed37aeff1e879eb191d1934b1fbfbf0dc892c90 WatchSource:0}: Error finding container 46a284ad4eb2bea2d22bc6973ed37aeff1e879eb191d1934b1fbfbf0dc892c90: Status 404 returned error can't find the container with id 46a284ad4eb2bea2d22bc6973ed37aeff1e879eb191d1934b1fbfbf0dc892c90
Jul 21 20:41:01.647275 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:01.647234    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk" event={"ID":"9652b851-dcff-4565-8524-277ff0ec643a","Type":"ContainerStarted","Data":"46a284ad4eb2bea2d22bc6973ed37aeff1e879eb191d1934b1fbfbf0dc892c90"}
Jul 21 20:41:02.010005 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:02.009949    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:02.083631 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:02.083567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:03.020510 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:03.020471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:03.125791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:03.125754    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:03.997476 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:03.997433    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:04.062721 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:04.062686    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:05.293975 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.293938    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:05.392698 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.392669    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:05.487217 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.487189    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:05.561272 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.561248    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:41:05.563322 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.563305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/37d8148e-f700-4a83-924f-a60ce06386e9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bvwwn\" (UID: \"37d8148e-f700-4a83-924f-a60ce06386e9\") " pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:41:05.661735 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.661697    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl" event={"ID":"2bf8996a-81f7-490c-acc0-a201ff2a902e","Type":"ContainerStarted","Data":"cae09ab391ef60f3a393758cd96a9a6bb09321d1172d8d7131d61e86104dbcf4"}
Jul 21 20:41:05.663559 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.663534    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk" event={"ID":"9652b851-dcff-4565-8524-277ff0ec643a","Type":"ContainerStarted","Data":"954947bef004a0eeaf366e0cef61d9f2be05efbe58af2e32aa3d0a8ea2497fd6"}
Jul 21 20:41:05.663661 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.663573    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk" event={"ID":"9652b851-dcff-4565-8524-277ff0ec643a","Type":"ContainerStarted","Data":"7d5f13f7c4a4dfb5c4e431f3ffcf91bc2d8a3ffc94f9ed7c0bd6041fe413e2a9"}
Jul 21 20:41:05.663661 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.663585    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk" event={"ID":"9652b851-dcff-4565-8524-277ff0ec643a","Type":"ContainerStarted","Data":"e95ca02d26dfcda3f4f3b3acda7f4fa3872fe39fe8679b1d3463e662c6dfd817"}
Jul 21 20:41:05.668281 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.668263    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bvwwn"
Jul 21 20:41:05.679787 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.679747    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-plspl" podStartSLOduration=33.134491112 podStartE2EDuration="37.679733351s" totalImagesPullingTime="4.545242239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:05.67940501 +0000 UTC m=+92.889433432" watchObservedRunningTime="2026-07-21 20:41:05.679733351 +0000 UTC m=+92.889761830"
Jul 21 20:41:05.698482 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.698442    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84bc6dc864-cdhqk" podStartSLOduration=1.287831515 podStartE2EDuration="5.698427924s" totalImagesPullingTime="4.410596409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:41:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:05.697494341 +0000 UTC m=+92.907522776" watchObservedRunningTime="2026-07-21 20:41:05.698427924 +0000 UTC m=+92.908456345"
Jul 21 20:41:05.798874 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.798843    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:05.815306 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:05.815270    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bvwwn"]
Jul 21 20:41:05.819034 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:41:05.818981    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d8148e_f700_4a83_924f_a60ce06386e9.slice/crio-f10d32d168769affee8fe0e5de95cd8d3ea00b1ea9d94b2504c8eebd93c605ed WatchSource:0}: Error finding container f10d32d168769affee8fe0e5de95cd8d3ea00b1ea9d94b2504c8eebd93c605ed: Status 404 returned error can't find the container with id f10d32d168769affee8fe0e5de95cd8d3ea00b1ea9d94b2504c8eebd93c605ed
Jul 21 20:41:06.111100 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:06.111009    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:06.211840 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:06.211798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:06.669020 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:06.668926    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bvwwn" event={"ID":"37d8148e-f700-4a83-924f-a60ce06386e9","Type":"ContainerStarted","Data":"f10d32d168769affee8fe0e5de95cd8d3ea00b1ea9d94b2504c8eebd93c605ed"}
Jul 21 20:41:08.103230 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:08.103195    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:08.204928 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:08.204891    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:09.489712 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:09.489677    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:09.582070 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:09.582028    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:09.831501 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:09.831397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:10.687672 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:10.687644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bvwwn" event={"ID":"37d8148e-f700-4a83-924f-a60ce06386e9","Type":"ContainerStarted","Data":"dad5fe3d11398da8660506eb2475cbc671e8042a216b8e7da3e7d28fb60d7006"}
Jul 21 20:41:10.772563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:10.772525    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:12.516686 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:12.516644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:41:12.519422 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:12.519393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"image-registry-59b8cd454d-pxwjg\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") " pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:41:12.617894 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:12.617846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:41:12.620542 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:12.620513    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2879e13a-d9ac-4b68-8353-c8f2d717423f-metrics-tls\") pod \"dns-default-7qx6q\" (UID: \"2879e13a-d9ac-4b68-8353-c8f2d717423f\") " pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:41:12.808629 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:12.808554    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9scjn\""
Jul 21 20:41:12.816860 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:12.816828    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:41:12.839645 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:12.839617    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lm6sh\""
Jul 21 20:41:12.846682 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:12.846653    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:41:13.045735 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:13.045697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:13.393915 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:13.393878    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:13.486613 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:13.486576    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:14.406888 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:14.406855    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7qx6q"]
Jul 21 20:41:14.410278 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:41:14.410255    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2879e13a_d9ac_4b68_8353_c8f2d717423f.slice/crio-be2889f9e1d7f0a5a1bd7db4f742e5cde79b91da74faec8a47043f20ea27959f WatchSource:0}: Error finding container be2889f9e1d7f0a5a1bd7db4f742e5cde79b91da74faec8a47043f20ea27959f: Status 404 returned error can't find the container with id be2889f9e1d7f0a5a1bd7db4f742e5cde79b91da74faec8a47043f20ea27959f
Jul 21 20:41:14.426475 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:14.426453    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-59b8cd454d-pxwjg"]
Jul 21 20:41:14.428822 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:41:14.428801    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff55187b_7adc_4356_84cb_8c3830f02ed7.slice/crio-5b95a0383b5ba9c0fd2a2266acf7a7dad9ce868d2d36055d7b0bd79bba4f92b5 WatchSource:0}: Error finding container 5b95a0383b5ba9c0fd2a2266acf7a7dad9ce868d2d36055d7b0bd79bba4f92b5: Status 404 returned error can't find the container with id 5b95a0383b5ba9c0fd2a2266acf7a7dad9ce868d2d36055d7b0bd79bba4f92b5
Jul 21 20:41:14.704484 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:14.704441    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg" event={"ID":"ff55187b-7adc-4356-84cb-8c3830f02ed7","Type":"ContainerStarted","Data":"bd8e6b8bb653961a56b23f9837e59dc00ed7d6e17ca7a0279665e2dbf3dd6766"}
Jul 21 20:41:14.704670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:14.704514    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:41:14.704670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:14.704535    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg" event={"ID":"ff55187b-7adc-4356-84cb-8c3830f02ed7","Type":"ContainerStarted","Data":"5b95a0383b5ba9c0fd2a2266acf7a7dad9ce868d2d36055d7b0bd79bba4f92b5"}
Jul 21 20:41:14.707953 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:14.707922    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7qx6q" event={"ID":"2879e13a-d9ac-4b68-8353-c8f2d717423f","Type":"ContainerStarted","Data":"be2889f9e1d7f0a5a1bd7db4f742e5cde79b91da74faec8a47043f20ea27959f"}
Jul 21 20:41:14.710169 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:14.710144    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bvwwn" event={"ID":"37d8148e-f700-4a83-924f-a60ce06386e9","Type":"ContainerStarted","Data":"3cea31196a60f1dfe80cada98a4492247c58684b1cb831b1d2ee0a2d781f00c2"}
Jul 21 20:41:14.725709 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:14.725660    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg" podStartSLOduration=101.725647277 podStartE2EDuration="1m41.725647277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:14.724730801 +0000 UTC m=+101.934759225" watchObservedRunningTime="2026-07-21 20:41:14.725647277 +0000 UTC m=+101.935675698"
Jul 21 20:41:15.604224 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:15.604189    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8cksp"
Jul 21 20:41:16.073469 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:16.073433    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:16.181008 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:16.180944    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:16.720862 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:16.720817    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7qx6q" event={"ID":"2879e13a-d9ac-4b68-8353-c8f2d717423f","Type":"ContainerStarted","Data":"3ff355cd1f3c236272b38517ad8104a3ac4a37cf483ff8d7e888402ecc4cf62e"}
Jul 21 20:41:16.721432 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:16.720927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7qx6q" event={"ID":"2879e13a-d9ac-4b68-8353-c8f2d717423f","Type":"ContainerStarted","Data":"fc433289a1a195e4a3e10ad836f235029150a064e2920db26bc9d70830ae9475"}
Jul 21 20:41:16.721786 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:16.721630    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:41:16.743575 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:16.743522    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7qx6q" podStartSLOduration=66.894560007 podStartE2EDuration="1m8.743509683s" totalImagesPullingTime="1.848949676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:16.741816098 +0000 UTC m=+103.951844538" watchObservedRunningTime="2026-07-21 20:41:16.743509683 +0000 UTC m=+103.953538103"
Jul 21 20:41:18.185724 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:18.185682    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:18.259884 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:18.259839    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:19.038801 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:19.038709    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:19.138486 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:19.138448    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:19.733023 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:19.732974    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bvwwn" event={"ID":"37d8148e-f700-4a83-924f-a60ce06386e9","Type":"ContainerStarted","Data":"ab154e8271b4f56c430c4224cd2f1c8a96b36ae18b904a0bba38bd58df0edda5"}
Jul 21 20:41:19.774273 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:19.774227    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bvwwn" podStartSLOduration=34.989104664 podStartE2EDuration="47.774211044s" totalImagesPullingTime="12.78510638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 20:40:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:19.771009211 +0000 UTC m=+106.981037647" watchObservedRunningTime="2026-07-21 20:41:19.774211044 +0000 UTC m=+106.984239465"
Jul 21 20:41:19.851812 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:19.851773    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:20.014570 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:20.014485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:24.015506 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:24.015474    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59b8cd454d-pxwjg"]
Jul 21 20:41:24.185927 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:24.185887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:24.270762 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:24.270677    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:25.103897 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:25.103865    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:26.134679 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.134645    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:26.235570 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.235533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:26.714429 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.714389    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:26.874444 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.874409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:26.962412 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.962377    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rcb59"]
Jul 21 20:41:26.965710 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.965652    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:26.968459 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.968439    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 20:41:26.968571 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.968459    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 20:41:26.968741 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.968726    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j5gnj\""
Jul 21 20:41:26.969636 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.969618    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 20:41:26.969740 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.969725    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 20:41:26.995454 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:26.995380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:27.030184 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030153    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5s88\" (UniqueName: \"kubernetes.io/projected/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-kube-api-access-v5s88\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.030320 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030187    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-sys\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.030320 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030204    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-root\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.030320 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-wtmp\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.030320 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030296    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-metrics-client-ca\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.030502 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030356    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.030502 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-textfile\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.030502 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030411    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-tls\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.030502 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.030434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.110544 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.110481    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:27.131431 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131398    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131577 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131436    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5s88\" (UniqueName: \"kubernetes.io/projected/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-kube-api-access-v5s88\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131577 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-sys\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131577 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131521    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-sys\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131738 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-root\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131738 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131621    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-wtmp\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131738 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-root\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131738 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-metrics-client-ca\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131933 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131746    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131933 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131769    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-textfile\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131933 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131795    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-tls\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131933 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131852    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-wtmp\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.131933 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:41:27.131893    2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 20:41:27.132208 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.131947    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-accelerators-collector-config\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.132208 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:41:27.131957    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-tls podName:cbdbf8ca-3542-475f-b293-f66fd9eb2f79 nodeName:}" failed. No retries permitted until 2026-07-21 20:41:27.631941069 +0000 UTC m=+114.841969483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-tls") pod "node-exporter-rcb59" (UID: "cbdbf8ca-3542-475f-b293-f66fd9eb2f79") : secret "node-exporter-tls" not found
Jul 21 20:41:27.132208 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.132124    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-textfile\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.132350 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.132277    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-metrics-client-ca\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.134076 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.134057    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.140594 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.140572    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5s88\" (UniqueName: \"kubernetes.io/projected/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-kube-api-access-v5s88\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.211645 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.211612    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:27.403548 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.403452    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:27.613022 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.612966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:27.635217 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.635190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-tls\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.637549 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.637525    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf8ca-3542-475f-b293-f66fd9eb2f79-node-exporter-tls\") pod \"node-exporter-rcb59\" (UID: \"cbdbf8ca-3542-475f-b293-f66fd9eb2f79\") " pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.729629 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.729597    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7qx6q"
Jul 21 20:41:27.875356 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.875320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rcb59"
Jul 21 20:41:27.885275 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.885242    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:27.887161 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:41:27.887137    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbdbf8ca_3542_475f_b293_f66fd9eb2f79.slice/crio-735c757dfd4bee42f4510ecee4cfdbcbaa8e1d82b3f1276f0e9563c399e6f4c0 WatchSource:0}: Error finding container 735c757dfd4bee42f4510ecee4cfdbcbaa8e1d82b3f1276f0e9563c399e6f4c0: Status 404 returned error can't find the container with id 735c757dfd4bee42f4510ecee4cfdbcbaa8e1d82b3f1276f0e9563c399e6f4c0
Jul 21 20:41:27.974195 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:27.974158    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:28.090021 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:28.089919    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:28.197193 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:28.197157    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:28.646832 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:28.646775    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:28.765527 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:28.765486    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rcb59" event={"ID":"cbdbf8ca-3542-475f-b293-f66fd9eb2f79","Type":"ContainerStarted","Data":"735c757dfd4bee42f4510ecee4cfdbcbaa8e1d82b3f1276f0e9563c399e6f4c0"}
Jul 21 20:41:29.770185 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:29.770145    2567 generic.go:356] "Generic (PLEG): container finished" podID="cbdbf8ca-3542-475f-b293-f66fd9eb2f79" containerID="c1c2cf46ce43fd86f3f1a2db37506b49d841f61c8be26fde35bd2b797d44e8dd" exitCode=0
Jul 21 20:41:29.770567 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:29.770214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rcb59" event={"ID":"cbdbf8ca-3542-475f-b293-f66fd9eb2f79","Type":"ContainerDied","Data":"c1c2cf46ce43fd86f3f1a2db37506b49d841f61c8be26fde35bd2b797d44e8dd"}
Jul 21 20:41:30.233816 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:30.233776    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:30.342824 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:30.342789    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:30.558481 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:30.558400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:30.775156 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:30.775121    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rcb59" event={"ID":"cbdbf8ca-3542-475f-b293-f66fd9eb2f79","Type":"ContainerStarted","Data":"9ec5f35e8f9d19fffeb672c6f798115d384f15c5023c5108d5255b63ea7d84df"}
Jul 21 20:41:30.775549 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:30.775183    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rcb59" event={"ID":"cbdbf8ca-3542-475f-b293-f66fd9eb2f79","Type":"ContainerStarted","Data":"8ea5257bd6587a2105601993cf42c06dc76cf9d3d943182fad7557e1339ff9c2"}
Jul 21 20:41:30.820269 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:30.820174    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rcb59" podStartSLOduration=3.897723642 podStartE2EDuration="4.820157677s" totalImagesPullingTime="859.447509ms" totalInitContainerRuntime="62.986526ms" isStatefulPod=true podCreationTimestamp="2026-07-21 20:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:30.8187089 +0000 UTC m=+118.028737341" watchObservedRunningTime="2026-07-21 20:41:30.820157677 +0000 UTC m=+118.030186100"
Jul 21 20:41:30.866735 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:30.866702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:31.208474 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:31.208441    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:31.309595 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:31.309556    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:31.771824 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:31.771790    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:31.931301 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:31.931264    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:32.032048 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:32.031943    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:32.283918 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:32.283834    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:33.599616 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:33.599579    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:34.021130 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:34.021102    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:41:35.221054 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:35.221004    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:35.283485 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:35.283451    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:36.882555 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:36.882519    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:36.984806 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:36.984772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:41.210714 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:41.210674    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:41.314874 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:41.314837    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:41.415403 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:41.415363    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:42.409020 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:42.408968    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:42.579636 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:42.579598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:43.054173 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:43.054136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:41:43.056365 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:43.056340    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf884bc1-53a5-47ab-98cd-3959ded9e6e7-metrics-certs\") pod \"network-metrics-daemon-tms9z\" (UID: \"cf884bc1-53a5-47ab-98cd-3959ded9e6e7\") " pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:41:43.333356 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:43.333281    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jpthn\""
Jul 21 20:41:43.341487 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:43.341468    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tms9z"
Jul 21 20:41:43.494132 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:43.494100    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tms9z"]
Jul 21 20:41:43.497372 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:41:43.497343    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf884bc1_53a5_47ab_98cd_3959ded9e6e7.slice/crio-23b5955528cae18e49a9756de924e8b7ef7bc6314710b4ea302e0ba4c80c2e42 WatchSource:0}: Error finding container 23b5955528cae18e49a9756de924e8b7ef7bc6314710b4ea302e0ba4c80c2e42: Status 404 returned error can't find the container with id 23b5955528cae18e49a9756de924e8b7ef7bc6314710b4ea302e0ba4c80c2e42
Jul 21 20:41:43.817776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:43.817738    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tms9z" event={"ID":"cf884bc1-53a5-47ab-98cd-3959ded9e6e7","Type":"ContainerStarted","Data":"23b5955528cae18e49a9756de924e8b7ef7bc6314710b4ea302e0ba4c80c2e42"}
Jul 21 20:41:45.827643 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:45.827594    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tms9z" event={"ID":"cf884bc1-53a5-47ab-98cd-3959ded9e6e7","Type":"ContainerStarted","Data":"6503e04a090ff5a8abca1c446731324a657143d50f80df71f8e69c5245b6dc1f"}
Jul 21 20:41:45.828083 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:45.827659    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tms9z" event={"ID":"cf884bc1-53a5-47ab-98cd-3959ded9e6e7","Type":"ContainerStarted","Data":"d82edfdce21eddea422d2772a3fdcfb95c1d5527e708f5d0feb8756053b07e82"}
Jul 21 20:41:45.845725 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:45.845675    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tms9z" podStartSLOduration=131.238590441 podStartE2EDuration="2m12.845661066s" totalImagesPullingTime="1.607070625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:41:45.844243475 +0000 UTC m=+133.054271917" watchObservedRunningTime="2026-07-21 20:41:45.845661066 +0000 UTC m=+133.055689487"
Jul 21 20:41:45.899796 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:45.899764    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:45.999172 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:45.999137    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:46.099390 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:46.099300    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:46.377782 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:46.377686    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:47.191443 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:47.191385    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:47.284771 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:47.284732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:41:49.034903 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:49.034862    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg" podUID="ff55187b-7adc-4356-84cb-8c3830f02ed7" containerName="registry" containerID="cri-o://bd8e6b8bb653961a56b23f9837e59dc00ed7d6e17ca7a0279665e2dbf3dd6766" gracePeriod=30
Jul 21 20:41:49.841556 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:49.841527    2567 generic.go:356] "Generic (PLEG): container finished" podID="ff55187b-7adc-4356-84cb-8c3830f02ed7" containerID="bd8e6b8bb653961a56b23f9837e59dc00ed7d6e17ca7a0279665e2dbf3dd6766" exitCode=0
Jul 21 20:41:49.841671 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:49.841585    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg" event={"ID":"ff55187b-7adc-4356-84cb-8c3830f02ed7","Type":"ContainerDied","Data":"bd8e6b8bb653961a56b23f9837e59dc00ed7d6e17ca7a0279665e2dbf3dd6766"}
Jul 21 20:41:49.841671 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:49.841621    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg" event={"ID":"ff55187b-7adc-4356-84cb-8c3830f02ed7","Type":"ContainerDied","Data":"5b95a0383b5ba9c0fd2a2266acf7a7dad9ce868d2d36055d7b0bd79bba4f92b5"}
Jul 21 20:41:49.841671 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:49.841635    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b95a0383b5ba9c0fd2a2266acf7a7dad9ce868d2d36055d7b0bd79bba4f92b5"
Jul 21 20:41:49.850606 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:49.850588    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:41:50.011564 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.011479    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") pod \"ff55187b-7adc-4356-84cb-8c3830f02ed7\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") "
Jul 21 20:41:50.011564 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.011531    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-bound-sa-token\") pod \"ff55187b-7adc-4356-84cb-8c3830f02ed7\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") "
Jul 21 20:41:50.011791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.011574    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-trusted-ca\") pod \"ff55187b-7adc-4356-84cb-8c3830f02ed7\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") "
Jul 21 20:41:50.011791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.011620    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-certificates\") pod \"ff55187b-7adc-4356-84cb-8c3830f02ed7\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") "
Jul 21 20:41:50.011791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.011644    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-installation-pull-secrets\") pod \"ff55187b-7adc-4356-84cb-8c3830f02ed7\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") "
Jul 21 20:41:50.011791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.011729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-image-registry-private-configuration\") pod \"ff55187b-7adc-4356-84cb-8c3830f02ed7\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") "
Jul 21 20:41:50.012022 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.011816    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbst2\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-kube-api-access-sbst2\") pod \"ff55187b-7adc-4356-84cb-8c3830f02ed7\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") "
Jul 21 20:41:50.012022 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.011844    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ff55187b-7adc-4356-84cb-8c3830f02ed7-ca-trust-extracted\") pod \"ff55187b-7adc-4356-84cb-8c3830f02ed7\" (UID: \"ff55187b-7adc-4356-84cb-8c3830f02ed7\") "
Jul 21 20:41:50.012145 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.012055    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ff55187b-7adc-4356-84cb-8c3830f02ed7" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 20:41:50.012336 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.012307    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ff55187b-7adc-4356-84cb-8c3830f02ed7" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 20:41:50.014166 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.014132    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ff55187b-7adc-4356-84cb-8c3830f02ed7" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 20:41:50.014166 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.014149    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ff55187b-7adc-4356-84cb-8c3830f02ed7" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:41:50.014332 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.014243    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ff55187b-7adc-4356-84cb-8c3830f02ed7" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 20:41:50.014332 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.014283    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ff55187b-7adc-4356-84cb-8c3830f02ed7" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:41:50.014777 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.014752    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-kube-api-access-sbst2" (OuterVolumeSpecName: "kube-api-access-sbst2") pod "ff55187b-7adc-4356-84cb-8c3830f02ed7" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7"). InnerVolumeSpecName "kube-api-access-sbst2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:41:50.021107 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.021084    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff55187b-7adc-4356-84cb-8c3830f02ed7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ff55187b-7adc-4356-84cb-8c3830f02ed7" (UID: "ff55187b-7adc-4356-84cb-8c3830f02ed7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:41:50.112829 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.112803    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-certificates\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:41:50.112829 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.112827    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-installation-pull-secrets\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:41:50.113194 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.112839    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ff55187b-7adc-4356-84cb-8c3830f02ed7-image-registry-private-configuration\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:41:50.113194 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.112848    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbst2\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-kube-api-access-sbst2\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:41:50.113194 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.112857    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ff55187b-7adc-4356-84cb-8c3830f02ed7-ca-trust-extracted\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:41:50.113194 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.112866    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-registry-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:41:50.113194 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.112875    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ff55187b-7adc-4356-84cb-8c3830f02ed7-bound-sa-token\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:41:50.113194 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.112882    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ff55187b-7adc-4356-84cb-8c3830f02ed7-trusted-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:41:50.848520 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.847688    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-59b8cd454d-pxwjg"
Jul 21 20:41:50.871607 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.871576    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-59b8cd454d-pxwjg"]
Jul 21 20:41:50.875595 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:50.875576    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-59b8cd454d-pxwjg"]
Jul 21 20:41:51.309969 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:51.309934    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff55187b-7adc-4356-84cb-8c3830f02ed7" path="/var/lib/kubelet/pods/ff55187b-7adc-4356-84cb-8c3830f02ed7/volumes"
Jul 21 20:41:57.872845 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:57.872808    2567 generic.go:356] "Generic (PLEG): container finished" podID="5503f039-d91f-464c-8731-8dbbecc97c04" containerID="a70efb1f158d9029a1e2b84a83e66273914dd27d43fb9d7b823933dd9f1a1aac" exitCode=0
Jul 21 20:41:57.873235 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:57.872884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lk8hn" event={"ID":"5503f039-d91f-464c-8731-8dbbecc97c04","Type":"ContainerDied","Data":"a70efb1f158d9029a1e2b84a83e66273914dd27d43fb9d7b823933dd9f1a1aac"}
Jul 21 20:41:57.873282 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:57.873251    2567 scope.go:117] "RemoveContainer" containerID="a70efb1f158d9029a1e2b84a83e66273914dd27d43fb9d7b823933dd9f1a1aac"
Jul 21 20:41:58.878487 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:58.878456    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-lk8hn" event={"ID":"5503f039-d91f-464c-8731-8dbbecc97c04","Type":"ContainerStarted","Data":"10bd26daaa22fd87ac75794c552f8c1a4383a8d530d68a321a6da9b36cc2027c"}
Jul 21 20:41:59.882568 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:59.882530    2567 generic.go:356] "Generic (PLEG): container finished" podID="79c9b2be-4e0d-4273-b125-1144be8c52de" containerID="55fa7d6243969f306bf9a39adaeeaee7e1b556dfd185e1f679d253bdbd596b47" exitCode=0
Jul 21 20:41:59.883193 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:59.882591    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5" event={"ID":"79c9b2be-4e0d-4273-b125-1144be8c52de","Type":"ContainerDied","Data":"55fa7d6243969f306bf9a39adaeeaee7e1b556dfd185e1f679d253bdbd596b47"}
Jul 21 20:41:59.883193 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:41:59.882962    2567 scope.go:117] "RemoveContainer" containerID="55fa7d6243969f306bf9a39adaeeaee7e1b556dfd185e1f679d253bdbd596b47"
Jul 21 20:42:00.888559 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:42:00.888521    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-nkkh5" event={"ID":"79c9b2be-4e0d-4273-b125-1144be8c52de","Type":"ContainerStarted","Data":"c4743ba5cdc03d954d3ca3299a9d0c90363823093f457e0ff55561b008049917"}
Jul 21 20:42:02.136933 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:42:02.136878    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:42:02.239256 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:42:02.239219    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:42:07.315247 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:42:07.315207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:42:46.573163 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:42:46.573120    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:42:46.682188 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:42:46.682158    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:42:56.262185 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:42:56.262150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:42:56.345731 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:42:56.345690    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:01.880454 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:01.880419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:01.978221 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:01.978182    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:06.058218 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:06.058175    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:06.228899 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:06.228841    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:06.559790 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:06.559756    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:06.659889 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:06.659845    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:13.912047 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:13.911973    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:14.011579 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:14.011540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:16.847360 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:16.847325    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:16.971806 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:16.971771    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:17.180831 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:17.180799    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:17.280920 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:17.280882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:17.380928 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:17.380888    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:28.734344 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:28.734309    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:43.349741 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:43.349703    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:43.447000 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:43.446944    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:44.369222 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.369188    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"]
Jul 21 20:43:44.369579 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.369434    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff55187b-7adc-4356-84cb-8c3830f02ed7" containerName="registry"
Jul 21 20:43:44.369579 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.369445    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff55187b-7adc-4356-84cb-8c3830f02ed7" containerName="registry"
Jul 21 20:43:44.369579 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.369494    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff55187b-7adc-4356-84cb-8c3830f02ed7" containerName="registry"
Jul 21 20:43:44.372289 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.372272    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.375666 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.375643    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vfsg5\""
Jul 21 20:43:44.376230 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.376210    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 20:43:44.377271 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.377250    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 20:43:44.393428 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.393402    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"]
Jul 21 20:43:44.426705 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.426678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxjkz\" (UniqueName: \"kubernetes.io/projected/bebb1d97-19a6-4407-9c17-42b2fa309aab-kube-api-access-mxjkz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.426840 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.426720    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.426840 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.426776    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.527119 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.527088    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.527292 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.527136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxjkz\" (UniqueName: \"kubernetes.io/projected/bebb1d97-19a6-4407-9c17-42b2fa309aab-kube-api-access-mxjkz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.527361 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.527318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.527553 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.527532    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.527627 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.527613    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.541234 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.541214    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxjkz\" (UniqueName: \"kubernetes.io/projected/bebb1d97-19a6-4407-9c17-42b2fa309aab-kube-api-access-mxjkz\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.680665 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.680633    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:43:44.823524 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:44.823491    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"]
Jul 21 20:43:44.826763 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:43:44.826739    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebb1d97_19a6_4407_9c17_42b2fa309aab.slice/crio-7186a1f8bb49ce70236fde377d29fa42aeb0af171067ed124b4d7a9fa60e4d7c WatchSource:0}: Error finding container 7186a1f8bb49ce70236fde377d29fa42aeb0af171067ed124b4d7a9fa60e4d7c: Status 404 returned error can't find the container with id 7186a1f8bb49ce70236fde377d29fa42aeb0af171067ed124b4d7a9fa60e4d7c
Jul 21 20:43:45.247757 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:45.247719    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f" event={"ID":"bebb1d97-19a6-4407-9c17-42b2fa309aab","Type":"ContainerStarted","Data":"7186a1f8bb49ce70236fde377d29fa42aeb0af171067ed124b4d7a9fa60e4d7c"}
Jul 21 20:43:51.555047 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:51.555009    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:43:52.274637 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:52.274554    2567 generic.go:356] "Generic (PLEG): container finished" podID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerID="cba7ef8e9988e45e9177ae579630376a9e277d683be06aa96a45571899ab4e68" exitCode=0
Jul 21 20:43:52.274637 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:52.274604    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f" event={"ID":"bebb1d97-19a6-4407-9c17-42b2fa309aab","Type":"ContainerDied","Data":"cba7ef8e9988e45e9177ae579630376a9e277d683be06aa96a45571899ab4e68"}
Jul 21 20:43:55.289684 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:55.289646    2567 generic.go:356] "Generic (PLEG): container finished" podID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerID="61989ae9cbea8a6ff016eb0c3ffe4791c10c89b944eb3baacc30aa0c9dcc00c2" exitCode=0
Jul 21 20:43:55.290148 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:43:55.289708    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f" event={"ID":"bebb1d97-19a6-4407-9c17-42b2fa309aab","Type":"ContainerDied","Data":"61989ae9cbea8a6ff016eb0c3ffe4791c10c89b944eb3baacc30aa0c9dcc00c2"}
Jul 21 20:44:03.322534 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:03.322495    2567 generic.go:356] "Generic (PLEG): container finished" podID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerID="4efe21711dde3a9d9a92b6ce09d7790009fbf387cb8c869d4d63a47f36ad62ce" exitCode=0
Jul 21 20:44:03.322534 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:03.322525    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f" event={"ID":"bebb1d97-19a6-4407-9c17-42b2fa309aab","Type":"ContainerDied","Data":"4efe21711dde3a9d9a92b6ce09d7790009fbf387cb8c869d4d63a47f36ad62ce"}
Jul 21 20:44:04.447051 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.447029    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:44:04.570228 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.570188    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-util\") pod \"bebb1d97-19a6-4407-9c17-42b2fa309aab\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") "
Jul 21 20:44:04.570416 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.570259    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-bundle\") pod \"bebb1d97-19a6-4407-9c17-42b2fa309aab\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") "
Jul 21 20:44:04.570416 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.570298    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxjkz\" (UniqueName: \"kubernetes.io/projected/bebb1d97-19a6-4407-9c17-42b2fa309aab-kube-api-access-mxjkz\") pod \"bebb1d97-19a6-4407-9c17-42b2fa309aab\" (UID: \"bebb1d97-19a6-4407-9c17-42b2fa309aab\") "
Jul 21 20:44:04.571103 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.571066    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-bundle" (OuterVolumeSpecName: "bundle") pod "bebb1d97-19a6-4407-9c17-42b2fa309aab" (UID: "bebb1d97-19a6-4407-9c17-42b2fa309aab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:44:04.572555 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.572524    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bebb1d97-19a6-4407-9c17-42b2fa309aab-kube-api-access-mxjkz" (OuterVolumeSpecName: "kube-api-access-mxjkz") pod "bebb1d97-19a6-4407-9c17-42b2fa309aab" (UID: "bebb1d97-19a6-4407-9c17-42b2fa309aab"). InnerVolumeSpecName "kube-api-access-mxjkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:44:04.577170 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.577101    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-util" (OuterVolumeSpecName: "util") pod "bebb1d97-19a6-4407-9c17-42b2fa309aab" (UID: "bebb1d97-19a6-4407-9c17-42b2fa309aab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:44:04.671702 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.671665    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-util\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:04.671702 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.671695    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bebb1d97-19a6-4407-9c17-42b2fa309aab-bundle\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:04.671702 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:04.671706    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxjkz\" (UniqueName: \"kubernetes.io/projected/bebb1d97-19a6-4407-9c17-42b2fa309aab-kube-api-access-mxjkz\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:05.330394 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:05.330361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f" event={"ID":"bebb1d97-19a6-4407-9c17-42b2fa309aab","Type":"ContainerDied","Data":"7186a1f8bb49ce70236fde377d29fa42aeb0af171067ed124b4d7a9fa60e4d7c"}
Jul 21 20:44:05.330394 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:05.330382    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540j5v5f"
Jul 21 20:44:05.330555 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:05.330444    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7186a1f8bb49ce70236fde377d29fa42aeb0af171067ed124b4d7a9fa60e4d7c"
Jul 21 20:44:10.734670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:10.734636    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:11.097714 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:11.097627    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:12.911288 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.911254    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"]
Jul 21 20:44:12.911653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.911508    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerName="util"
Jul 21 20:44:12.911653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.911519    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerName="util"
Jul 21 20:44:12.911653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.911529    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerName="pull"
Jul 21 20:44:12.911653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.911534    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerName="pull"
Jul 21 20:44:12.911653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.911549    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerName="extract"
Jul 21 20:44:12.911653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.911554    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerName="extract"
Jul 21 20:44:12.911653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.911592    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bebb1d97-19a6-4407-9c17-42b2fa309aab" containerName="extract"
Jul 21 20:44:12.915430 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.915410    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"
Jul 21 20:44:12.918019 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.917978    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 21 20:44:12.918019 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.918006    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 21 20:44:12.918323 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.918303    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gms8h\""
Jul 21 20:44:12.931752 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:12.931729    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"]
Jul 21 20:44:13.027203 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:13.027163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ddrl\" (UniqueName: \"kubernetes.io/projected/64c2d4fb-3daa-4f8b-a022-3985eb64a55e-kube-api-access-2ddrl\") pod \"cert-manager-operator-controller-manager-55c444bf86-rsj7c\" (UID: \"64c2d4fb-3daa-4f8b-a022-3985eb64a55e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"
Jul 21 20:44:13.027203 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:13.027199    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64c2d4fb-3daa-4f8b-a022-3985eb64a55e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rsj7c\" (UID: \"64c2d4fb-3daa-4f8b-a022-3985eb64a55e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"
Jul 21 20:44:13.127602 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:13.127562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddrl\" (UniqueName: \"kubernetes.io/projected/64c2d4fb-3daa-4f8b-a022-3985eb64a55e-kube-api-access-2ddrl\") pod \"cert-manager-operator-controller-manager-55c444bf86-rsj7c\" (UID: \"64c2d4fb-3daa-4f8b-a022-3985eb64a55e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"
Jul 21 20:44:13.127602 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:13.127598    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64c2d4fb-3daa-4f8b-a022-3985eb64a55e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rsj7c\" (UID: \"64c2d4fb-3daa-4f8b-a022-3985eb64a55e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"
Jul 21 20:44:13.127938 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:13.127922    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64c2d4fb-3daa-4f8b-a022-3985eb64a55e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-rsj7c\" (UID: \"64c2d4fb-3daa-4f8b-a022-3985eb64a55e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"
Jul 21 20:44:13.138556 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:13.138524    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ddrl\" (UniqueName: \"kubernetes.io/projected/64c2d4fb-3daa-4f8b-a022-3985eb64a55e-kube-api-access-2ddrl\") pod \"cert-manager-operator-controller-manager-55c444bf86-rsj7c\" (UID: \"64c2d4fb-3daa-4f8b-a022-3985eb64a55e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"
Jul 21 20:44:13.223799 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:13.223767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"
Jul 21 20:44:13.402671 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:13.402637    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c"]
Jul 21 20:44:13.406014 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:44:13.405971    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c2d4fb_3daa_4f8b_a022_3985eb64a55e.slice/crio-aefe8f9d8ae2ab7019b7c122509001b8d53d4635f4a8e374c5734666c51e2eee WatchSource:0}: Error finding container aefe8f9d8ae2ab7019b7c122509001b8d53d4635f4a8e374c5734666c51e2eee: Status 404 returned error can't find the container with id aefe8f9d8ae2ab7019b7c122509001b8d53d4635f4a8e374c5734666c51e2eee
Jul 21 20:44:14.364446 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:14.364409    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c" event={"ID":"64c2d4fb-3daa-4f8b-a022-3985eb64a55e","Type":"ContainerStarted","Data":"aefe8f9d8ae2ab7019b7c122509001b8d53d4635f4a8e374c5734666c51e2eee"}
Jul 21 20:44:14.787874 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:14.787834    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:14.909773 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:14.909733    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:15.369742 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:15.369709    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c" event={"ID":"64c2d4fb-3daa-4f8b-a022-3985eb64a55e","Type":"ContainerStarted","Data":"867d9d56d346a294e6f361dfe4d3c72c2c5cefca370420927c11087d199c9dd3"}
Jul 21 20:44:15.400964 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:15.400883    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-rsj7c" podStartSLOduration=1.669266632 podStartE2EDuration="3.400868048s" totalImagesPullingTime="1.731601416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:44:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:44:15.400119763 +0000 UTC m=+282.610148207" watchObservedRunningTime="2026-07-21 20:44:15.400868048 +0000 UTC m=+282.610896469"
Jul 21 20:44:15.828790 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:15.828752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:15.973074 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:15.973028    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:16.468256 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.468225    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"]
Jul 21 20:44:16.471356 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.471338    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.475125 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.475100    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 20:44:16.475412 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.475393    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 20:44:16.476102 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.476083    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vfsg5\""
Jul 21 20:44:16.491338 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.491302    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"]
Jul 21 20:44:16.554286 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.554252    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjhmp\" (UniqueName: \"kubernetes.io/projected/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-kube-api-access-cjhmp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.554425 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.554313    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.554425 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.554373    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.654733 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.654696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.654733 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.654732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.654959 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.654780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjhmp\" (UniqueName: \"kubernetes.io/projected/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-kube-api-access-cjhmp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.655176 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.655153    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.655220 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.655166    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.669529 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.669503    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:16.672463 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.672443    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjhmp\" (UniqueName: \"kubernetes.io/projected/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-kube-api-access-cjhmp\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.783648 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.783582    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:16.944047 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:16.944016    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"]
Jul 21 20:44:16.946571 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:44:16.946542    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c3847a_f4f7_41cf_84e9_cc6b1e15f0aa.slice/crio-0b55e2e66c012404ab07d56256af004b1a25f6397bd833cf517fd93349cc2da0 WatchSource:0}: Error finding container 0b55e2e66c012404ab07d56256af004b1a25f6397bd833cf517fd93349cc2da0: Status 404 returned error can't find the container with id 0b55e2e66c012404ab07d56256af004b1a25f6397bd833cf517fd93349cc2da0
Jul 21 20:44:17.132240 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:17.132213    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:17.263836 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:17.263802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:17.378438 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:17.378350    2567 generic.go:356] "Generic (PLEG): container finished" podID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerID="f1cd4e1390a73c151d8b8110655730682f9ad3fae7ad502586ea2f0c55f194f1" exitCode=0
Jul 21 20:44:17.378438 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:17.378423    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876" event={"ID":"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa","Type":"ContainerDied","Data":"f1cd4e1390a73c151d8b8110655730682f9ad3fae7ad502586ea2f0c55f194f1"}
Jul 21 20:44:17.378657 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:17.378458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876" event={"ID":"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa","Type":"ContainerStarted","Data":"0b55e2e66c012404ab07d56256af004b1a25f6397bd833cf517fd93349cc2da0"}
Jul 21 20:44:19.387695 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:19.387669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876" event={"ID":"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa","Type":"ContainerStarted","Data":"adf71ce3953f216bf641665acca5362ea84f76f27e430c890fd6d5ccec515014"}
Jul 21 20:44:19.554342 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:19.554307    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:19.741088 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:19.741055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:20.393518 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.393485    2567 generic.go:356] "Generic (PLEG): container finished" podID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerID="adf71ce3953f216bf641665acca5362ea84f76f27e430c890fd6d5ccec515014" exitCode=0
Jul 21 20:44:20.393518 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.393520    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876" event={"ID":"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa","Type":"ContainerDied","Data":"adf71ce3953f216bf641665acca5362ea84f76f27e430c890fd6d5ccec515014"}
Jul 21 20:44:20.738500 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.738463    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"]
Jul 21 20:44:20.741571 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.741552    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"
Jul 21 20:44:20.745290 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.745267    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 20:44:20.750129 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.750110    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 20:44:20.750207 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.750110    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-gpnz7\""
Jul 21 20:44:20.762478 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.762450    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"]
Jul 21 20:44:20.784507 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.784476    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1148ca35-2247-4f9f-b6f0-c4dd95c3be1f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gx6gl\" (UID: \"1148ca35-2247-4f9f-b6f0-c4dd95c3be1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"
Jul 21 20:44:20.784507 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.784521    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4bms\" (UniqueName: \"kubernetes.io/projected/1148ca35-2247-4f9f-b6f0-c4dd95c3be1f-kube-api-access-l4bms\") pod \"cert-manager-cainjector-559cdd8c87-gx6gl\" (UID: \"1148ca35-2247-4f9f-b6f0-c4dd95c3be1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"
Jul 21 20:44:20.885603 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.885560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4bms\" (UniqueName: \"kubernetes.io/projected/1148ca35-2247-4f9f-b6f0-c4dd95c3be1f-kube-api-access-l4bms\") pod \"cert-manager-cainjector-559cdd8c87-gx6gl\" (UID: \"1148ca35-2247-4f9f-b6f0-c4dd95c3be1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"
Jul 21 20:44:20.885807 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.885641    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1148ca35-2247-4f9f-b6f0-c4dd95c3be1f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gx6gl\" (UID: \"1148ca35-2247-4f9f-b6f0-c4dd95c3be1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"
Jul 21 20:44:20.908902 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.908874    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1148ca35-2247-4f9f-b6f0-c4dd95c3be1f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gx6gl\" (UID: \"1148ca35-2247-4f9f-b6f0-c4dd95c3be1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"
Jul 21 20:44:20.909375 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:20.909355    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4bms\" (UniqueName: \"kubernetes.io/projected/1148ca35-2247-4f9f-b6f0-c4dd95c3be1f-kube-api-access-l4bms\") pod \"cert-manager-cainjector-559cdd8c87-gx6gl\" (UID: \"1148ca35-2247-4f9f-b6f0-c4dd95c3be1f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"
Jul 21 20:44:21.050402 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:21.050313    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"
Jul 21 20:44:21.184260 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:21.184231    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl"]
Jul 21 20:44:21.186571 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:44:21.186541    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1148ca35_2247_4f9f_b6f0_c4dd95c3be1f.slice/crio-9ac1c6d558c31019217ab76670910dea19ef5f45cf4ea35fc289e09080de2e18 WatchSource:0}: Error finding container 9ac1c6d558c31019217ab76670910dea19ef5f45cf4ea35fc289e09080de2e18: Status 404 returned error can't find the container with id 9ac1c6d558c31019217ab76670910dea19ef5f45cf4ea35fc289e09080de2e18
Jul 21 20:44:21.399849 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:21.399761    2567 generic.go:356] "Generic (PLEG): container finished" podID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerID="5dc037fe087dc04bd3fb83d4db7641dc0dc46f07b1993f8d6e580b413e69774e" exitCode=0
Jul 21 20:44:21.399849 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:21.399840    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876" event={"ID":"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa","Type":"ContainerDied","Data":"5dc037fe087dc04bd3fb83d4db7641dc0dc46f07b1993f8d6e580b413e69774e"}
Jul 21 20:44:21.400842 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:21.400815    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl" event={"ID":"1148ca35-2247-4f9f-b6f0-c4dd95c3be1f","Type":"ContainerStarted","Data":"9ac1c6d558c31019217ab76670910dea19ef5f45cf4ea35fc289e09080de2e18"}
Jul 21 20:44:21.580170 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:21.580134    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:21.722586 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:21.722549    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:22.545391 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.545357    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:22.599018 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.598970    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-util\") pod \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") "
Jul 21 20:44:22.599180 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.599090    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-bundle\") pod \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") "
Jul 21 20:44:22.599180 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.599164    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjhmp\" (UniqueName: \"kubernetes.io/projected/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-kube-api-access-cjhmp\") pod \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\" (UID: \"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa\") "
Jul 21 20:44:22.599530 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.599504    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-bundle" (OuterVolumeSpecName: "bundle") pod "27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" (UID: "27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:44:22.601405 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.601377    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-kube-api-access-cjhmp" (OuterVolumeSpecName: "kube-api-access-cjhmp") pod "27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" (UID: "27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa"). InnerVolumeSpecName "kube-api-access-cjhmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:44:22.604832 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.604805    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-util" (OuterVolumeSpecName: "util") pod "27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" (UID: "27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:44:22.700322 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.700282    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjhmp\" (UniqueName: \"kubernetes.io/projected/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-kube-api-access-cjhmp\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:22.700322 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.700318    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-util\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:22.700322 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:22.700332    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa-bundle\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:23.411803 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:23.411770    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876" event={"ID":"27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa","Type":"ContainerDied","Data":"0b55e2e66c012404ab07d56256af004b1a25f6397bd833cf517fd93349cc2da0"}
Jul 21 20:44:23.412014 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:23.411809    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flc876"
Jul 21 20:44:23.412014 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:23.411822    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b55e2e66c012404ab07d56256af004b1a25f6397bd833cf517fd93349cc2da0"
Jul 21 20:44:24.417411 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:24.417374    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl" event={"ID":"1148ca35-2247-4f9f-b6f0-c4dd95c3be1f","Type":"ContainerStarted","Data":"fed91215237e8003e2f4ac99443e1302fd7b45a4c714dd6abc3879d48f36db95"}
Jul 21 20:44:24.434631 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:24.434584    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gx6gl" podStartSLOduration=1.773773721 podStartE2EDuration="4.434558335s" totalImagesPullingTime="2.660784614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:44:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:44:24.433637854 +0000 UTC m=+291.643666279" watchObservedRunningTime="2026-07-21 20:44:24.434558335 +0000 UTC m=+291.644586750"
Jul 21 20:44:24.635102 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:24.635071    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:24.770071 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:24.769971    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:27.076954 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.076894    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:27.738880 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.738847    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qt82d"]
Jul 21 20:44:27.739109 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.739096    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerName="util"
Jul 21 20:44:27.739162 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.739111    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerName="util"
Jul 21 20:44:27.739162 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.739120    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerName="extract"
Jul 21 20:44:27.739162 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.739124    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerName="extract"
Jul 21 20:44:27.739162 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.739139    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerName="pull"
Jul 21 20:44:27.739162 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.739144    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerName="pull"
Jul 21 20:44:27.739308 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.739180    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c3847a-f4f7-41cf-84e9-cc6b1e15f0aa" containerName="extract"
Jul 21 20:44:27.741931 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.741914    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qt82d"
Jul 21 20:44:27.744390 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.744369    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-jwlz7\""
Jul 21 20:44:27.757576 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.757551    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qt82d"]
Jul 21 20:44:27.839594 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.839563    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cae2b78-cbd0-43ea-9c70-e7bba6ff743a-bound-sa-token\") pod \"cert-manager-55b4d954d5-qt82d\" (UID: \"4cae2b78-cbd0-43ea-9c70-e7bba6ff743a\") " pod="cert-manager/cert-manager-55b4d954d5-qt82d"
Jul 21 20:44:27.839721 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.839626    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm7gz\" (UniqueName: \"kubernetes.io/projected/4cae2b78-cbd0-43ea-9c70-e7bba6ff743a-kube-api-access-bm7gz\") pod \"cert-manager-55b4d954d5-qt82d\" (UID: \"4cae2b78-cbd0-43ea-9c70-e7bba6ff743a\") " pod="cert-manager/cert-manager-55b4d954d5-qt82d"
Jul 21 20:44:27.940295 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.940263    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm7gz\" (UniqueName: \"kubernetes.io/projected/4cae2b78-cbd0-43ea-9c70-e7bba6ff743a-kube-api-access-bm7gz\") pod \"cert-manager-55b4d954d5-qt82d\" (UID: \"4cae2b78-cbd0-43ea-9c70-e7bba6ff743a\") " pod="cert-manager/cert-manager-55b4d954d5-qt82d"
Jul 21 20:44:27.940421 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.940310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cae2b78-cbd0-43ea-9c70-e7bba6ff743a-bound-sa-token\") pod \"cert-manager-55b4d954d5-qt82d\" (UID: \"4cae2b78-cbd0-43ea-9c70-e7bba6ff743a\") " pod="cert-manager/cert-manager-55b4d954d5-qt82d"
Jul 21 20:44:27.948887 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.948849    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm7gz\" (UniqueName: \"kubernetes.io/projected/4cae2b78-cbd0-43ea-9c70-e7bba6ff743a-kube-api-access-bm7gz\") pod \"cert-manager-55b4d954d5-qt82d\" (UID: \"4cae2b78-cbd0-43ea-9c70-e7bba6ff743a\") " pod="cert-manager/cert-manager-55b4d954d5-qt82d"
Jul 21 20:44:27.949490 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:27.949467    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4cae2b78-cbd0-43ea-9c70-e7bba6ff743a-bound-sa-token\") pod \"cert-manager-55b4d954d5-qt82d\" (UID: \"4cae2b78-cbd0-43ea-9c70-e7bba6ff743a\") " pod="cert-manager/cert-manager-55b4d954d5-qt82d"
Jul 21 20:44:28.050148 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:28.050063    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-qt82d"
Jul 21 20:44:28.169454 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:28.169421    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-qt82d"]
Jul 21 20:44:28.172552 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:44:28.172524    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cae2b78_cbd0_43ea_9c70_e7bba6ff743a.slice/crio-e1259c17daa88cba61f5b1920e79d6f879653700ba6d5f94e36574f1f5b4ba2d WatchSource:0}: Error finding container e1259c17daa88cba61f5b1920e79d6f879653700ba6d5f94e36574f1f5b4ba2d: Status 404 returned error can't find the container with id e1259c17daa88cba61f5b1920e79d6f879653700ba6d5f94e36574f1f5b4ba2d
Jul 21 20:44:28.434599 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:28.434561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qt82d" event={"ID":"4cae2b78-cbd0-43ea-9c70-e7bba6ff743a","Type":"ContainerStarted","Data":"fb64b9366306fc027d0007941f5e66e67a8b84470984c65d69caca7de95653c9"}
Jul 21 20:44:28.434761 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:28.434609    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-qt82d" event={"ID":"4cae2b78-cbd0-43ea-9c70-e7bba6ff743a","Type":"ContainerStarted","Data":"e1259c17daa88cba61f5b1920e79d6f879653700ba6d5f94e36574f1f5b4ba2d"}
Jul 21 20:44:28.452915 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:28.452864    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-qt82d" podStartSLOduration=1.452846148 podStartE2EDuration="1.452846148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:44:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:44:28.452511683 +0000 UTC m=+295.662540134" watchObservedRunningTime="2026-07-21 20:44:28.452846148 +0000 UTC m=+295.662874565"
Jul 21 20:44:28.648931 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:28.648899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:28.752154 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:28.752064    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:29.248735 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:29.248703    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:33.214199 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:33.214167    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:44:33.219974 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:33.219949    2567 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 20:44:33.220229 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:33.220212    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:44:34.704981 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:34.704931    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:34.800274 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:34.800233    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:36.468823 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.468780    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:36.566655 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.566622    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"]
Jul 21 20:44:36.588456 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.588432    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"]
Jul 21 20:44:36.588578 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.588532    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.591210 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.591188    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 21 20:44:36.591352 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.591231    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 21 20:44:36.591608 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.591593    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vfsg5\""
Jul 21 20:44:36.702496 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.702455    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.702646 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.702554    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.702646 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.702595    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g9cf\" (UniqueName: \"kubernetes.io/projected/42e0c49a-1b23-4724-ad57-7fe452172c71-kube-api-access-7g9cf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.803715 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.803621    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.803715 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.803670    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g9cf\" (UniqueName: \"kubernetes.io/projected/42e0c49a-1b23-4724-ad57-7fe452172c71-kube-api-access-7g9cf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.803937 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.803739    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.804061 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.804043    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.804170 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.804138    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.812936 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.812913    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g9cf\" (UniqueName: \"kubernetes.io/projected/42e0c49a-1b23-4724-ad57-7fe452172c71-kube-api-access-7g9cf\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:36.897215 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:36.897184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:37.043193 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:37.043138    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"]
Jul 21 20:44:37.045648 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:44:37.045619    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e0c49a_1b23_4724_ad57_7fe452172c71.slice/crio-5564cfd6cb2778bbd348e621f30fd333fc6907a534ad7f4d1baaf77c0c935f99 WatchSource:0}: Error finding container 5564cfd6cb2778bbd348e621f30fd333fc6907a534ad7f4d1baaf77c0c935f99: Status 404 returned error can't find the container with id 5564cfd6cb2778bbd348e621f30fd333fc6907a534ad7f4d1baaf77c0c935f99
Jul 21 20:44:37.474790 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:37.474755    2567 generic.go:356] "Generic (PLEG): container finished" podID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerID="24f52663720c193fee7186c9d724e27c5909f764491554ccc0340de43652eefe" exitCode=0
Jul 21 20:44:37.475173 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:37.474824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6" event={"ID":"42e0c49a-1b23-4724-ad57-7fe452172c71","Type":"ContainerDied","Data":"24f52663720c193fee7186c9d724e27c5909f764491554ccc0340de43652eefe"}
Jul 21 20:44:37.475173 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:37.474871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6" event={"ID":"42e0c49a-1b23-4724-ad57-7fe452172c71","Type":"ContainerStarted","Data":"5564cfd6cb2778bbd348e621f30fd333fc6907a534ad7f4d1baaf77c0c935f99"}
Jul 21 20:44:37.475304 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:37.475289    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 20:44:39.483431 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:39.483396    2567 generic.go:356] "Generic (PLEG): container finished" podID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerID="79e4f3ba113b7e946a3326035e3e28d5306e4ad487db05179d80fe2efc6a82f6" exitCode=0
Jul 21 20:44:39.483859 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:39.483480    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6" event={"ID":"42e0c49a-1b23-4724-ad57-7fe452172c71","Type":"ContainerDied","Data":"79e4f3ba113b7e946a3326035e3e28d5306e4ad487db05179d80fe2efc6a82f6"}
Jul 21 20:44:40.489023 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:40.488969    2567 generic.go:356] "Generic (PLEG): container finished" podID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerID="3ecf37d656866746526c2b64492bc17d58a9d1ab08c0f7a48659b50700b34783" exitCode=0
Jul 21 20:44:40.489023 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:40.489020    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6" event={"ID":"42e0c49a-1b23-4724-ad57-7fe452172c71","Type":"ContainerDied","Data":"3ecf37d656866746526c2b64492bc17d58a9d1ab08c0f7a48659b50700b34783"}
Jul 21 20:44:41.622510 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.622480    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:41.739612 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.739567    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g9cf\" (UniqueName: \"kubernetes.io/projected/42e0c49a-1b23-4724-ad57-7fe452172c71-kube-api-access-7g9cf\") pod \"42e0c49a-1b23-4724-ad57-7fe452172c71\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") "
Jul 21 20:44:41.739612 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.739621    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-bundle\") pod \"42e0c49a-1b23-4724-ad57-7fe452172c71\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") "
Jul 21 20:44:41.739910 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.739677    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-util\") pod \"42e0c49a-1b23-4724-ad57-7fe452172c71\" (UID: \"42e0c49a-1b23-4724-ad57-7fe452172c71\") "
Jul 21 20:44:41.740386 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.740345    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-bundle" (OuterVolumeSpecName: "bundle") pod "42e0c49a-1b23-4724-ad57-7fe452172c71" (UID: "42e0c49a-1b23-4724-ad57-7fe452172c71"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:44:41.741714 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.741693    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42e0c49a-1b23-4724-ad57-7fe452172c71-kube-api-access-7g9cf" (OuterVolumeSpecName: "kube-api-access-7g9cf") pod "42e0c49a-1b23-4724-ad57-7fe452172c71" (UID: "42e0c49a-1b23-4724-ad57-7fe452172c71"). InnerVolumeSpecName "kube-api-access-7g9cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:44:41.745273 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.745186    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-util" (OuterVolumeSpecName: "util") pod "42e0c49a-1b23-4724-ad57-7fe452172c71" (UID: "42e0c49a-1b23-4724-ad57-7fe452172c71"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:44:41.840261 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.840211    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-util\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:41.840261 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.840256    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g9cf\" (UniqueName: \"kubernetes.io/projected/42e0c49a-1b23-4724-ad57-7fe452172c71-kube-api-access-7g9cf\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:41.840261 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:41.840270    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/42e0c49a-1b23-4724-ad57-7fe452172c71-bundle\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:44:42.498883 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:42.498859    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6"
Jul 21 20:44:42.499053 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:42.498849    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6lcc6" event={"ID":"42e0c49a-1b23-4724-ad57-7fe452172c71","Type":"ContainerDied","Data":"5564cfd6cb2778bbd348e621f30fd333fc6907a534ad7f4d1baaf77c0c935f99"}
Jul 21 20:44:42.499053 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:42.498977    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5564cfd6cb2778bbd348e621f30fd333fc6907a534ad7f4d1baaf77c0c935f99"
Jul 21 20:44:52.671594 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:52.671543    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:52.777601 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:52.777564    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:53.435841 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:53.435807    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:53.911459 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:53.911351    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:54.017554 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.017502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:54.091916 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.091882    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"]
Jul 21 20:44:54.092179 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.092166    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerName="extract"
Jul 21 20:44:54.092223 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.092180    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerName="extract"
Jul 21 20:44:54.092223 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.092192    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerName="pull"
Jul 21 20:44:54.092223 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.092198    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerName="pull"
Jul 21 20:44:54.092223 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.092209    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerName="util"
Jul 21 20:44:54.092223 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.092213    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerName="util"
Jul 21 20:44:54.092365 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.092268    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="42e0c49a-1b23-4724-ad57-7fe452172c71" containerName="extract"
Jul 21 20:44:54.098907 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.098886    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.101176 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.101154    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 20:44:54.102141 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.102123    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 20:44:54.102239 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.102189    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rvl9p\""
Jul 21 20:44:54.102302 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.102250    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 20:44:54.102456 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.102440    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 20:44:54.124776 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.124740    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"]
Jul 21 20:44:54.235530 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.235492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc6dl\" (UniqueName: \"kubernetes.io/projected/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-kube-api-access-bc6dl\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.235703 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.235540    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-manifests\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.235703 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.235612    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-tmp\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.235703 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.235647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.235703 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.235665    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-webhook-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.271700 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.271662    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:54.336108 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.336073    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc6dl\" (UniqueName: \"kubernetes.io/projected/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-kube-api-access-bc6dl\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.336279 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.336119    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-manifests\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.336279 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.336184    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-tmp\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.336279 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.336213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.336279 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.336235    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-webhook-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.336629 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.336606    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-manifests\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.338643 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.338614    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-tmp\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.338857 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.338836    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-webhook-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.338902 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.338836    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.345053 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.345033    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc6dl\" (UniqueName: \"kubernetes.io/projected/2b50bada-ba17-49dd-98c3-3f1a0b3d6255-kube-api-access-bc6dl\") pod \"opendatahub-operator-controller-manager-647bdf6675-bsdv5\" (UID: \"2b50bada-ba17-49dd-98c3-3f1a0b3d6255\") " pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.371177 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.371147    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:54.408865 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.408835    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:44:54.472459 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.472389    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:54.562609 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.562582    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"]
Jul 21 20:44:54.564940 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:44:54.564858    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b50bada_ba17_49dd_98c3_3f1a0b3d6255.slice/crio-badc4f98d3e2ec8e9326a5973bdcbf0b33bab3b3a355d6fd2ff5bc324530e605 WatchSource:0}: Error finding container badc4f98d3e2ec8e9326a5973bdcbf0b33bab3b3a355d6fd2ff5bc324530e605: Status 404 returned error can't find the container with id badc4f98d3e2ec8e9326a5973bdcbf0b33bab3b3a355d6fd2ff5bc324530e605
Jul 21 20:44:54.574795 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.574768    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:54.700797 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.700751    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:54.822833 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.822745    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:54.930515 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:54.930477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:55.549808 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:55.549768    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5" event={"ID":"2b50bada-ba17-49dd-98c3-3f1a0b3d6255","Type":"ContainerStarted","Data":"badc4f98d3e2ec8e9326a5973bdcbf0b33bab3b3a355d6fd2ff5bc324530e605"}
Jul 21 20:44:56.713395 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:56.713358    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:56.833618 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:56.833575    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:44:58.565644 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:58.565605    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5" event={"ID":"2b50bada-ba17-49dd-98c3-3f1a0b3d6255","Type":"ContainerStarted","Data":"6ee973ce933a49df240a2025ebbdfd25911a56c9afc4a83216921a4213b0a8c8"}
Jul 21 20:44:59.571631 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:59.571597    2567 generic.go:356] "Generic (PLEG): container finished" podID="2b50bada-ba17-49dd-98c3-3f1a0b3d6255" containerID="6ee973ce933a49df240a2025ebbdfd25911a56c9afc4a83216921a4213b0a8c8" exitCode=0
Jul 21 20:44:59.571631 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:44:59.571633    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5" event={"ID":"2b50bada-ba17-49dd-98c3-3f1a0b3d6255","Type":"ContainerDied","Data":"6ee973ce933a49df240a2025ebbdfd25911a56c9afc4a83216921a4213b0a8c8"}
Jul 21 20:45:00.578393 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:00.578360    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5" event={"ID":"2b50bada-ba17-49dd-98c3-3f1a0b3d6255","Type":"ContainerStarted","Data":"5fc415d89c3b20a5ff20324cc31b826e6ea1df815e95b431ac4b36f8e56709a6"}
Jul 21 20:45:00.578887 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:00.578463    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:45:00.614358 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:00.614299    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5" podStartSLOduration=2.021057136 podStartE2EDuration="6.614282429s" totalImagesPullingTime="3.544888636s" totalInitContainerRuntime="1.048336657s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:44:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:00.61174989 +0000 UTC m=+327.821778324" watchObservedRunningTime="2026-07-21 20:45:00.614282429 +0000 UTC m=+327.824310844"
Jul 21 20:45:05.678409 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:05.678371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:07.661344 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:07.661300    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:11.584515 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:11.584481    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-647bdf6675-bsdv5"
Jul 21 20:45:13.764221 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:13.764186    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:13.983977 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:13.983933    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:14.850435 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:14.850385    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:15.181558 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.181506    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"]
Jul 21 20:45:15.189311 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.189285    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.201709 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.201684    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 20:45:15.201830 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.201695    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 20:45:15.202027 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.202011    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gk2gw\""
Jul 21 20:45:15.205035 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.205017    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 20:45:15.212780 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.212762    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 20:45:15.219755 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.219735    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"]
Jul 21 20:45:15.221936 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.221913    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:15.293599 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.293562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8chd9\" (UniqueName: \"kubernetes.io/projected/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-kube-api-access-8chd9\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.293781 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.293657    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-tls-certs\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.293781 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.293688    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-tmp\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.394815 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.394778    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8chd9\" (UniqueName: \"kubernetes.io/projected/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-kube-api-access-8chd9\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.394997 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.394850    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-tls-certs\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.394997 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.394872    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-tmp\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.397177 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.397147    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-tmp\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.397384 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.397367    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-tls-certs\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.444234 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.444167    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8chd9\" (UniqueName: \"kubernetes.io/projected/4f5f8352-40f8-4213-ba2f-f1e4963e92e2-kube-api-access-8chd9\") pod \"kube-auth-proxy-65495b7cc9-8w4z4\" (UID: \"4f5f8352-40f8-4213-ba2f-f1e4963e92e2\") " pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.498226 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.498188    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"
Jul 21 20:45:15.533581 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.533540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:15.646648 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:45:15.646616    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f5f8352_40f8_4213_ba2f_f1e4963e92e2.slice/crio-992b63664b1dcd3e9548d174f66540e65d9dd4c4f878e61cbb8cfd56aeddee5f WatchSource:0}: Error finding container 992b63664b1dcd3e9548d174f66540e65d9dd4c4f878e61cbb8cfd56aeddee5f: Status 404 returned error can't find the container with id 992b63664b1dcd3e9548d174f66540e65d9dd4c4f878e61cbb8cfd56aeddee5f
Jul 21 20:45:15.647780 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.647758    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4"]
Jul 21 20:45:15.780815 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:15.780732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:16.005680 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:16.005642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:16.645270 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:16.645233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4" event={"ID":"4f5f8352-40f8-4213-ba2f-f1e4963e92e2","Type":"ContainerStarted","Data":"992b63664b1dcd3e9548d174f66540e65d9dd4c4f878e61cbb8cfd56aeddee5f"}
Jul 21 20:45:16.925282 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:16.925239    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:17.434622 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:17.434582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:17.989473 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:17.989415    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:18.089404 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:18.089366    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:18.653966 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:18.653927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4" event={"ID":"4f5f8352-40f8-4213-ba2f-f1e4963e92e2","Type":"ContainerStarted","Data":"93ac659078d910f7267a59061669b67c7a31ed7ec47688267a3d0dea0d56c026"}
Jul 21 20:45:18.677769 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:18.677720    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65495b7cc9-8w4z4" podStartSLOduration=0.807893861 podStartE2EDuration="3.677704174s" totalImagesPullingTime="2.869810313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:18.676293187 +0000 UTC m=+345.886321628" watchObservedRunningTime="2026-07-21 20:45:18.677704174 +0000 UTC m=+345.887732594"
Jul 21 20:45:19.371334 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:19.371290    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:23.614851 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.614808    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"]
Jul 21 20:45:23.618282 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.618259    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.621791 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.621769    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jd7dm\""
Jul 21 20:45:23.621920 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.621815    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 20:45:23.621920 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.621835    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 20:45:23.626799 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.622499    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 20:45:23.626799 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.622509    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 20:45:23.638067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.638041    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"]
Jul 21 20:45:23.656946 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.656915    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.657061 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.656973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.657061 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.657039    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.657150 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.657069    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.657150 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.657116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/66657f1f-bee9-4f60-bfc8-231a7e196eee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.657227 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.657148    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.657227 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.657179    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqfhl\" (UniqueName: \"kubernetes.io/projected/66657f1f-bee9-4f60-bfc8-231a7e196eee-kube-api-access-nqfhl\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.758077 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.758036    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.758077 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.758078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/66657f1f-bee9-4f60-bfc8-231a7e196eee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.758338 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.758100    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.758338 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.758136    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqfhl\" (UniqueName: \"kubernetes.io/projected/66657f1f-bee9-4f60-bfc8-231a7e196eee-kube-api-access-nqfhl\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.758338 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.758182    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.758338 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.758228    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.758338 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.758257    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.758830 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.758783    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.760490 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.760461    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.760490 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.760482    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/66657f1f-bee9-4f60-bfc8-231a7e196eee-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.760627 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.760514    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.760667 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.760632    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.772391 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.772367    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66657f1f-bee9-4f60-bfc8-231a7e196eee-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.772543 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.772523    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqfhl\" (UniqueName: \"kubernetes.io/projected/66657f1f-bee9-4f60-bfc8-231a7e196eee-kube-api-access-nqfhl\") pod \"istiod-openshift-gateway-94bb8fbfd-pxkdh\" (UID: \"66657f1f-bee9-4f60-bfc8-231a7e196eee\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.932421 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.932391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:23.943625 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:23.943571    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:24.067684 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:24.067633    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:24.095044 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:24.095015    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"]
Jul 21 20:45:24.099292 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:45:24.099267    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66657f1f_bee9_4f60_bfc8_231a7e196eee.slice/crio-746d44eebb8025b27428ccbe36c70bcb11ce6460621d5b69629aa1abd5e3ce62 WatchSource:0}: Error finding container 746d44eebb8025b27428ccbe36c70bcb11ce6460621d5b69629aa1abd5e3ce62: Status 404 returned error can't find the container with id 746d44eebb8025b27428ccbe36c70bcb11ce6460621d5b69629aa1abd5e3ce62
Jul 21 20:45:24.681723 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:24.681681    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh" event={"ID":"66657f1f-bee9-4f60-bfc8-231a7e196eee","Type":"ContainerStarted","Data":"746d44eebb8025b27428ccbe36c70bcb11ce6460621d5b69629aa1abd5e3ce62"}
Jul 21 20:45:25.413510 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:25.413469    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:25.746114 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:25.745975    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:25.968923 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:25.968882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:26.267292 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:26.267247    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 20:45:26.267387 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:26.267360    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 20:45:26.693975 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:26.693937    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh" event={"ID":"66657f1f-bee9-4f60-bfc8-231a7e196eee","Type":"ContainerStarted","Data":"2e6ebcd5b01906db6e003ff5a80d8193a5f1b60e2825fd30a35362f50f61eee7"}
Jul 21 20:45:26.694147 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:26.694024    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:26.775277 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:26.775207    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh" podStartSLOduration=1.609284685 podStartE2EDuration="3.775190303s" totalImagesPullingTime="2.165905618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:45:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:26.773372341 +0000 UTC m=+353.983400782" watchObservedRunningTime="2026-07-21 20:45:26.775190303 +0000 UTC m=+353.985218723"
Jul 21 20:45:27.699966 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:27.699941    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pxkdh"
Jul 21 20:45:27.733765 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:27.733722    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:27.961977 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:27.961900    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"]
Jul 21 20:45:27.965203 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:27.965183    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:27.968924 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:27.968906    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 21 20:45:27.976891 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:27.976873    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-zq24d\""
Jul 21 20:45:27.997718 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:27.993311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvnbr\" (UniqueName: \"kubernetes.io/projected/b84d8279-e5c7-46fd-b42c-f6e52922ed23-kube-api-access-hvnbr\") pod \"llmisvc-controller-manager-777c6bd7d5-j7w99\" (UID: \"b84d8279-e5c7-46fd-b42c-f6e52922ed23\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:27.997718 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:27.993438    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b84d8279-e5c7-46fd-b42c-f6e52922ed23-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-j7w99\" (UID: \"b84d8279-e5c7-46fd-b42c-f6e52922ed23\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:28.020491 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.020457    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"]
Jul 21 20:45:28.022529 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.022497    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:28.094266 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.094231    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvnbr\" (UniqueName: \"kubernetes.io/projected/b84d8279-e5c7-46fd-b42c-f6e52922ed23-kube-api-access-hvnbr\") pod \"llmisvc-controller-manager-777c6bd7d5-j7w99\" (UID: \"b84d8279-e5c7-46fd-b42c-f6e52922ed23\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:28.094413 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.094291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b84d8279-e5c7-46fd-b42c-f6e52922ed23-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-j7w99\" (UID: \"b84d8279-e5c7-46fd-b42c-f6e52922ed23\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:28.096685 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.096654    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b84d8279-e5c7-46fd-b42c-f6e52922ed23-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-j7w99\" (UID: \"b84d8279-e5c7-46fd-b42c-f6e52922ed23\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:28.110055 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.110028    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvnbr\" (UniqueName: \"kubernetes.io/projected/b84d8279-e5c7-46fd-b42c-f6e52922ed23-kube-api-access-hvnbr\") pod \"llmisvc-controller-manager-777c6bd7d5-j7w99\" (UID: \"b84d8279-e5c7-46fd-b42c-f6e52922ed23\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:28.144308 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.144275    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:28.274172 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.274094    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:28.439791 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:45:28.439752    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb84d8279_e5c7_46fd_b42c_f6e52922ed23.slice/crio-1d14c76ede6fa05c23282237ff5274f3633217229196093dbead7a8fbc56b35f WatchSource:0}: Error finding container 1d14c76ede6fa05c23282237ff5274f3633217229196093dbead7a8fbc56b35f: Status 404 returned error can't find the container with id 1d14c76ede6fa05c23282237ff5274f3633217229196093dbead7a8fbc56b35f
Jul 21 20:45:28.440161 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.440137    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"]
Jul 21 20:45:28.704142 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:28.704106    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99" event={"ID":"b84d8279-e5c7-46fd-b42c-f6e52922ed23","Type":"ContainerStarted","Data":"1d14c76ede6fa05c23282237ff5274f3633217229196093dbead7a8fbc56b35f"}
Jul 21 20:45:29.259021 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:29.258966    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:29.420042 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:29.420006    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:29.687727 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:29.687677    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:45:32.724721 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:32.724681    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99" event={"ID":"b84d8279-e5c7-46fd-b42c-f6e52922ed23","Type":"ContainerStarted","Data":"be7b20a5d502daeea2e05768c72d96d292e72fdf3ea5bd589d864e713a205bd3"}
Jul 21 20:45:32.725215 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:32.724764    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:45:32.761546 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:45:32.761494    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99" podStartSLOduration=2.16606361 podStartE2EDuration="5.761477852s" totalImagesPullingTime="3.595414242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:45:32.760424306 +0000 UTC m=+359.970452748" watchObservedRunningTime="2026-07-21 20:45:32.761477852 +0000 UTC m=+359.971506273"
Jul 21 20:46:01.728305 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:01.728250    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:01.828406 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:01.828371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:03.731319 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:03.731284    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-j7w99"
Jul 21 20:46:15.301724 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:15.301690    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:15.404517 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:15.404478    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:15.504144 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:15.504108    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:16.241420 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:16.241375    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:16.444465 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:16.444419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:28.040590 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.040557    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"]
Jul 21 20:46:28.043926 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.043909    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.046819 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.046792    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tlcgm\""
Jul 21 20:46:28.046917 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.046792    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 20:46:28.047632 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.047612    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 20:46:28.053044 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.053022    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"]
Jul 21 20:46:28.073059 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.073030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.073215 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.073065    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.073215 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.073102    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lsv2\" (UniqueName: \"kubernetes.io/projected/a7d9fdeb-f64c-4b40-a579-5c588fc20786-kube-api-access-8lsv2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.173618 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.173577    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.173618 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.173620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.173880 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.173728    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lsv2\" (UniqueName: \"kubernetes.io/projected/a7d9fdeb-f64c-4b40-a579-5c588fc20786-kube-api-access-8lsv2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.174096 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.174072    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.174096 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.174087    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.182387 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.182365    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lsv2\" (UniqueName: \"kubernetes.io/projected/a7d9fdeb-f64c-4b40-a579-5c588fc20786-kube-api-access-8lsv2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.353751 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.353661    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:28.491465 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.491426    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"]
Jul 21 20:46:28.494973 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:46:28.494947    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d9fdeb_f64c_4b40_a579_5c588fc20786.slice/crio-44e473000993e4c1f5cd4ef68a42c16f3241d799575733f6ccc2733c239f8be9 WatchSource:0}: Error finding container 44e473000993e4c1f5cd4ef68a42c16f3241d799575733f6ccc2733c239f8be9: Status 404 returned error can't find the container with id 44e473000993e4c1f5cd4ef68a42c16f3241d799575733f6ccc2733c239f8be9
Jul 21 20:46:28.648019 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.647936    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"]
Jul 21 20:46:28.651161 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.651146    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.659598 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.659568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"]
Jul 21 20:46:28.678263 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.678227    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdmxg\" (UniqueName: \"kubernetes.io/projected/a024eea2-3868-406d-aadd-30673217b8f1-kube-api-access-xdmxg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.678409 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.678280    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.678409 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.678378    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.779249 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.779205    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.779450 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.779279    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdmxg\" (UniqueName: \"kubernetes.io/projected/a024eea2-3868-406d-aadd-30673217b8f1-kube-api-access-xdmxg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.779450 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.779322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.779638 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.779615    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.779704 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.779666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.787786 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.787764    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdmxg\" (UniqueName: \"kubernetes.io/projected/a024eea2-3868-406d-aadd-30673217b8f1-kube-api-access-xdmxg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.961150 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.961113    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:28.967356 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.967319    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerID="5b12e552fc6be28662f3cb102b6a279b85e1b30b832ef0b689b3aab937f019ce" exitCode=0
Jul 21 20:46:28.967480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.967402    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh" event={"ID":"a7d9fdeb-f64c-4b40-a579-5c588fc20786","Type":"ContainerDied","Data":"5b12e552fc6be28662f3cb102b6a279b85e1b30b832ef0b689b3aab937f019ce"}
Jul 21 20:46:28.967480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:28.967451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh" event={"ID":"a7d9fdeb-f64c-4b40-a579-5c588fc20786","Type":"ContainerStarted","Data":"44e473000993e4c1f5cd4ef68a42c16f3241d799575733f6ccc2733c239f8be9"}
Jul 21 20:46:29.091497 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.091471    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"]
Jul 21 20:46:29.093239 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:46:29.093210    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda024eea2_3868_406d_aadd_30673217b8f1.slice/crio-aa82a6eb8f7d86d50f335a5dcb5d5bf844b2e1737ac2fbb8318c310423a84af8 WatchSource:0}: Error finding container aa82a6eb8f7d86d50f335a5dcb5d5bf844b2e1737ac2fbb8318c310423a84af8: Status 404 returned error can't find the container with id aa82a6eb8f7d86d50f335a5dcb5d5bf844b2e1737ac2fbb8318c310423a84af8
Jul 21 20:46:29.251545 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.251472    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"]
Jul 21 20:46:29.254675 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.254658    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.264410 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.264385    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"]
Jul 21 20:46:29.283053 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.283028    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.283160 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.283062    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjxl5\" (UniqueName: \"kubernetes.io/projected/4e842868-4f6c-4dbb-882c-e913c9f7b20b-kube-api-access-kjxl5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.283160 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.283092    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.383713 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.383676    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.383713 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.383716    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjxl5\" (UniqueName: \"kubernetes.io/projected/4e842868-4f6c-4dbb-882c-e913c9f7b20b-kube-api-access-kjxl5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.383929 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.383739    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.384192 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.384172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.384192 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.384185    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.392194 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.392172    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjxl5\" (UniqueName: \"kubernetes.io/projected/4e842868-4f6c-4dbb-882c-e913c9f7b20b-kube-api-access-kjxl5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.582408 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.582324    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:29.724745 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.723389    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"]
Jul 21 20:46:29.728765 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:46:29.728728    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e842868_4f6c_4dbb_882c_e913c9f7b20b.slice/crio-c7744e4d080a6a0bf9ba123a787a09f64507e9cc428a97468bb8df57d6023d33 WatchSource:0}: Error finding container c7744e4d080a6a0bf9ba123a787a09f64507e9cc428a97468bb8df57d6023d33: Status 404 returned error can't find the container with id c7744e4d080a6a0bf9ba123a787a09f64507e9cc428a97468bb8df57d6023d33
Jul 21 20:46:29.905902 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.905832    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"]
Jul 21 20:46:29.909305 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.909288    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:29.934528 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.934504    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"]
Jul 21 20:46:29.973890 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.973856    2567 generic.go:356] "Generic (PLEG): container finished" podID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerID="076ca6ab0f4b88a6cecab226e2fa296d6ef9e5e22199e7a8457769cf09dbd04a" exitCode=0
Jul 21 20:46:29.974077 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.973940    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj" event={"ID":"4e842868-4f6c-4dbb-882c-e913c9f7b20b","Type":"ContainerDied","Data":"076ca6ab0f4b88a6cecab226e2fa296d6ef9e5e22199e7a8457769cf09dbd04a"}
Jul 21 20:46:29.974077 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.974028    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj" event={"ID":"4e842868-4f6c-4dbb-882c-e913c9f7b20b","Type":"ContainerStarted","Data":"c7744e4d080a6a0bf9ba123a787a09f64507e9cc428a97468bb8df57d6023d33"}
Jul 21 20:46:29.975650 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.975627    2567 generic.go:356] "Generic (PLEG): container finished" podID="a024eea2-3868-406d-aadd-30673217b8f1" containerID="98e65ce175660a4b54c5333dc42d9d6112511087255fd45f6c427bcad32a6046" exitCode=0
Jul 21 20:46:29.975770 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.975702    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm" event={"ID":"a024eea2-3868-406d-aadd-30673217b8f1","Type":"ContainerDied","Data":"98e65ce175660a4b54c5333dc42d9d6112511087255fd45f6c427bcad32a6046"}
Jul 21 20:46:29.975770 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.975743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm" event={"ID":"a024eea2-3868-406d-aadd-30673217b8f1","Type":"ContainerStarted","Data":"aa82a6eb8f7d86d50f335a5dcb5d5bf844b2e1737ac2fbb8318c310423a84af8"}
Jul 21 20:46:29.989449 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.989415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:29.989575 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.989526    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:29.989637 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:29.989576    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzfd8\" (UniqueName: \"kubernetes.io/projected/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-kube-api-access-rzfd8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:30.090248 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.090224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:30.090363 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.090260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:30.090363 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.090287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzfd8\" (UniqueName: \"kubernetes.io/projected/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-kube-api-access-rzfd8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:30.090553 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.090531    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:30.090660 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.090639    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:30.104214 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.104089    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzfd8\" (UniqueName: \"kubernetes.io/projected/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-kube-api-access-rzfd8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:30.220787 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.220758    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:30.368419 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.368381    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"]
Jul 21 20:46:30.374225 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:46:30.374197    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4599dfc0_81db_4dee_9d05_0e6f4b91bdb6.slice/crio-2969d3ff73ea8e347963f39e77b7e14bc84d08b953b7bc8b424ba0d02bffd82a WatchSource:0}: Error finding container 2969d3ff73ea8e347963f39e77b7e14bc84d08b953b7bc8b424ba0d02bffd82a: Status 404 returned error can't find the container with id 2969d3ff73ea8e347963f39e77b7e14bc84d08b953b7bc8b424ba0d02bffd82a
Jul 21 20:46:30.981966 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.981898    2567 generic.go:356] "Generic (PLEG): container finished" podID="a024eea2-3868-406d-aadd-30673217b8f1" containerID="705fcfc8dc8be3de2d912a955b27e334f1080d95974b4fb8df59718366007657" exitCode=0
Jul 21 20:46:30.981966 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.981935    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm" event={"ID":"a024eea2-3868-406d-aadd-30673217b8f1","Type":"ContainerDied","Data":"705fcfc8dc8be3de2d912a955b27e334f1080d95974b4fb8df59718366007657"}
Jul 21 20:46:30.983520 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.983489    2567 generic.go:356] "Generic (PLEG): container finished" podID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerID="c79f4bbff3ab6be0b5c0e2ca8363551dcfcf7ebde75336ce919322234fa0a566" exitCode=0
Jul 21 20:46:30.983607 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.983564    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n" event={"ID":"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6","Type":"ContainerDied","Data":"c79f4bbff3ab6be0b5c0e2ca8363551dcfcf7ebde75336ce919322234fa0a566"}
Jul 21 20:46:30.983607 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.983596    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n" event={"ID":"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6","Type":"ContainerStarted","Data":"2969d3ff73ea8e347963f39e77b7e14bc84d08b953b7bc8b424ba0d02bffd82a"}
Jul 21 20:46:30.985343 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.985321    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerID="063cd28ba55237a58f8a07f431a05cbbd77a1efbdb768c0e4baefd7bc8b33021" exitCode=0
Jul 21 20:46:30.985443 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.985417    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh" event={"ID":"a7d9fdeb-f64c-4b40-a579-5c588fc20786","Type":"ContainerDied","Data":"063cd28ba55237a58f8a07f431a05cbbd77a1efbdb768c0e4baefd7bc8b33021"}
Jul 21 20:46:30.987550 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.987529    2567 generic.go:356] "Generic (PLEG): container finished" podID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerID="b66b351b7cf760b52b0848d2e04c2b78b1c696a9295571a416af789ccbf15a69" exitCode=0
Jul 21 20:46:30.987648 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:30.987561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj" event={"ID":"4e842868-4f6c-4dbb-882c-e913c9f7b20b","Type":"ContainerDied","Data":"b66b351b7cf760b52b0848d2e04c2b78b1c696a9295571a416af789ccbf15a69"}
Jul 21 20:46:31.994583 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:31.994553    2567 generic.go:356] "Generic (PLEG): container finished" podID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerID="c0ef4781f69d8782912518831f36fffabb4aa7e21818b6db1ba4e36a27abcdcc" exitCode=0
Jul 21 20:46:31.995032 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:31.994638    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj" event={"ID":"4e842868-4f6c-4dbb-882c-e913c9f7b20b","Type":"ContainerDied","Data":"c0ef4781f69d8782912518831f36fffabb4aa7e21818b6db1ba4e36a27abcdcc"}
Jul 21 20:46:31.996786 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:31.996766    2567 generic.go:356] "Generic (PLEG): container finished" podID="a024eea2-3868-406d-aadd-30673217b8f1" containerID="e1c38dc9b011a7a092de2ec7bbdcbd2fe4d52e16d7db4854ff49447a3afec147" exitCode=0
Jul 21 20:46:31.996908 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:31.996831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm" event={"ID":"a024eea2-3868-406d-aadd-30673217b8f1","Type":"ContainerDied","Data":"e1c38dc9b011a7a092de2ec7bbdcbd2fe4d52e16d7db4854ff49447a3afec147"}
Jul 21 20:46:31.999114 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:31.999090    2567 generic.go:356] "Generic (PLEG): container finished" podID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerID="5a772db65c071aa29b28b3c41156a9517f12491d1f6d6643f334fc6c40568824" exitCode=0
Jul 21 20:46:31.999207 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:31.999155    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh" event={"ID":"a7d9fdeb-f64c-4b40-a579-5c588fc20786","Type":"ContainerDied","Data":"5a772db65c071aa29b28b3c41156a9517f12491d1f6d6643f334fc6c40568824"}
Jul 21 20:46:33.005053 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.005014    2567 generic.go:356] "Generic (PLEG): container finished" podID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerID="a52275dc36c6d1a10e0d66b7391da09c0ffdce35dcbe4099387b3d4dae5e7f58" exitCode=0
Jul 21 20:46:33.005488 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.005107    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n" event={"ID":"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6","Type":"ContainerDied","Data":"a52275dc36c6d1a10e0d66b7391da09c0ffdce35dcbe4099387b3d4dae5e7f58"}
Jul 21 20:46:33.233742 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.233716    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:33.236603 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.236570    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:33.239862 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.239840    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:33.317130 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317071    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-bundle\") pod \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") "
Jul 21 20:46:33.317130 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317102    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-util\") pod \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") "
Jul 21 20:46:33.317130 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317123    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lsv2\" (UniqueName: \"kubernetes.io/projected/a7d9fdeb-f64c-4b40-a579-5c588fc20786-kube-api-access-8lsv2\") pod \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") "
Jul 21 20:46:33.317563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317183    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-util\") pod \"a024eea2-3868-406d-aadd-30673217b8f1\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") "
Jul 21 20:46:33.317563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317220    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-util\") pod \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\" (UID: \"a7d9fdeb-f64c-4b40-a579-5c588fc20786\") "
Jul 21 20:46:33.317563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317263    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdmxg\" (UniqueName: \"kubernetes.io/projected/a024eea2-3868-406d-aadd-30673217b8f1-kube-api-access-xdmxg\") pod \"a024eea2-3868-406d-aadd-30673217b8f1\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") "
Jul 21 20:46:33.317563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317288    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-bundle\") pod \"a024eea2-3868-406d-aadd-30673217b8f1\" (UID: \"a024eea2-3868-406d-aadd-30673217b8f1\") "
Jul 21 20:46:33.317563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317336    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-bundle\") pod \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") "
Jul 21 20:46:33.317563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317371    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjxl5\" (UniqueName: \"kubernetes.io/projected/4e842868-4f6c-4dbb-882c-e913c9f7b20b-kube-api-access-kjxl5\") pod \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\" (UID: \"4e842868-4f6c-4dbb-882c-e913c9f7b20b\") "
Jul 21 20:46:33.317952 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.317717    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-bundle" (OuterVolumeSpecName: "bundle") pod "a7d9fdeb-f64c-4b40-a579-5c588fc20786" (UID: "a7d9fdeb-f64c-4b40-a579-5c588fc20786"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:46:33.318186 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.318130    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-bundle" (OuterVolumeSpecName: "bundle") pod "a024eea2-3868-406d-aadd-30673217b8f1" (UID: "a024eea2-3868-406d-aadd-30673217b8f1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:46:33.318417 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.318375    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-bundle" (OuterVolumeSpecName: "bundle") pod "4e842868-4f6c-4dbb-882c-e913c9f7b20b" (UID: "4e842868-4f6c-4dbb-882c-e913c9f7b20b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:46:33.319728 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.319702    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7d9fdeb-f64c-4b40-a579-5c588fc20786-kube-api-access-8lsv2" (OuterVolumeSpecName: "kube-api-access-8lsv2") pod "a7d9fdeb-f64c-4b40-a579-5c588fc20786" (UID: "a7d9fdeb-f64c-4b40-a579-5c588fc20786"). InnerVolumeSpecName "kube-api-access-8lsv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:46:33.319866 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.319843    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e842868-4f6c-4dbb-882c-e913c9f7b20b-kube-api-access-kjxl5" (OuterVolumeSpecName: "kube-api-access-kjxl5") pod "4e842868-4f6c-4dbb-882c-e913c9f7b20b" (UID: "4e842868-4f6c-4dbb-882c-e913c9f7b20b"). InnerVolumeSpecName "kube-api-access-kjxl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:46:33.320317 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.320282    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a024eea2-3868-406d-aadd-30673217b8f1-kube-api-access-xdmxg" (OuterVolumeSpecName: "kube-api-access-xdmxg") pod "a024eea2-3868-406d-aadd-30673217b8f1" (UID: "a024eea2-3868-406d-aadd-30673217b8f1"). InnerVolumeSpecName "kube-api-access-xdmxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:46:33.323014 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.322979    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-util" (OuterVolumeSpecName: "util") pod "4e842868-4f6c-4dbb-882c-e913c9f7b20b" (UID: "4e842868-4f6c-4dbb-882c-e913c9f7b20b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:46:33.324141 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.324121    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-util" (OuterVolumeSpecName: "util") pod "a024eea2-3868-406d-aadd-30673217b8f1" (UID: "a024eea2-3868-406d-aadd-30673217b8f1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:46:33.324261 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.324232    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-util" (OuterVolumeSpecName: "util") pod "a7d9fdeb-f64c-4b40-a579-5c588fc20786" (UID: "a7d9fdeb-f64c-4b40-a579-5c588fc20786"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:46:33.417919 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417879    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-bundle\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:33.417919 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417912    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjxl5\" (UniqueName: \"kubernetes.io/projected/4e842868-4f6c-4dbb-882c-e913c9f7b20b-kube-api-access-kjxl5\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:33.417919 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417923    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-bundle\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:33.418175 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417931    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e842868-4f6c-4dbb-882c-e913c9f7b20b-util\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:33.418175 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417940    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lsv2\" (UniqueName: \"kubernetes.io/projected/a7d9fdeb-f64c-4b40-a579-5c588fc20786-kube-api-access-8lsv2\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:33.418175 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417948    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-util\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:33.418175 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417956    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7d9fdeb-f64c-4b40-a579-5c588fc20786-util\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:33.418175 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417964    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdmxg\" (UniqueName: \"kubernetes.io/projected/a024eea2-3868-406d-aadd-30673217b8f1-kube-api-access-xdmxg\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:33.418175 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:33.417971    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a024eea2-3868-406d-aadd-30673217b8f1-bundle\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:34.011006 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.010961    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj"
Jul 21 20:46:34.011426 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.010965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj" event={"ID":"4e842868-4f6c-4dbb-882c-e913c9f7b20b","Type":"ContainerDied","Data":"c7744e4d080a6a0bf9ba123a787a09f64507e9cc428a97468bb8df57d6023d33"}
Jul 21 20:46:34.011426 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.011084    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7744e4d080a6a0bf9ba123a787a09f64507e9cc428a97468bb8df57d6023d33"
Jul 21 20:46:34.012922 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.012898    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm" event={"ID":"a024eea2-3868-406d-aadd-30673217b8f1","Type":"ContainerDied","Data":"aa82a6eb8f7d86d50f335a5dcb5d5bf844b2e1737ac2fbb8318c310423a84af8"}
Jul 21 20:46:34.013072 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.012933    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa82a6eb8f7d86d50f335a5dcb5d5bf844b2e1737ac2fbb8318c310423a84af8"
Jul 21 20:46:34.013072 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.012940    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm"
Jul 21 20:46:34.015091 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.015064    2567 generic.go:356] "Generic (PLEG): container finished" podID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerID="198ab96092e80f59699e081aa7aa1dd9a2def8ea76364c4137d219226455601a" exitCode=0
Jul 21 20:46:34.015189 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.015135    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n" event={"ID":"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6","Type":"ContainerDied","Data":"198ab96092e80f59699e081aa7aa1dd9a2def8ea76364c4137d219226455601a"}
Jul 21 20:46:34.017172 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.017149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh" event={"ID":"a7d9fdeb-f64c-4b40-a579-5c588fc20786","Type":"ContainerDied","Data":"44e473000993e4c1f5cd4ef68a42c16f3241d799575733f6ccc2733c239f8be9"}
Jul 21 20:46:34.017274 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.017188    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44e473000993e4c1f5cd4ef68a42c16f3241d799575733f6ccc2733c239f8be9"
Jul 21 20:46:34.017274 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:34.017158    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh"
Jul 21 20:46:35.146880 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.146854    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:35.232282 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.232246    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-util\") pod \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") "
Jul 21 20:46:35.232463 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.232311    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-bundle\") pod \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") "
Jul 21 20:46:35.232463 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.232357    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzfd8\" (UniqueName: \"kubernetes.io/projected/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-kube-api-access-rzfd8\") pod \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\" (UID: \"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6\") "
Jul 21 20:46:35.232760 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.232736    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-bundle" (OuterVolumeSpecName: "bundle") pod "4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" (UID: "4599dfc0-81db-4dee-9d05-0e6f4b91bdb6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:46:35.234431 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.234410    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-kube-api-access-rzfd8" (OuterVolumeSpecName: "kube-api-access-rzfd8") pod "4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" (UID: "4599dfc0-81db-4dee-9d05-0e6f4b91bdb6"). InnerVolumeSpecName "kube-api-access-rzfd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:46:35.237661 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.237633    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-util" (OuterVolumeSpecName: "util") pod "4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" (UID: "4599dfc0-81db-4dee-9d05-0e6f4b91bdb6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:46:35.333810 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.333783    2567 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-bundle\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:35.333810 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.333809    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzfd8\" (UniqueName: \"kubernetes.io/projected/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-kube-api-access-rzfd8\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:35.333951 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:35.333818    2567 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4599dfc0-81db-4dee-9d05-0e6f4b91bdb6-util\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:46:36.028492 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:36.028462    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n"
Jul 21 20:46:36.028651 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:36.028493    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n" event={"ID":"4599dfc0-81db-4dee-9d05-0e6f4b91bdb6","Type":"ContainerDied","Data":"2969d3ff73ea8e347963f39e77b7e14bc84d08b953b7bc8b424ba0d02bffd82a"}
Jul 21 20:46:36.028651 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:36.028539    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2969d3ff73ea8e347963f39e77b7e14bc84d08b953b7bc8b424ba0d02bffd82a"
Jul 21 20:46:42.351457 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:42.351412    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:42.563873 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:42.563842    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:43.108719 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:43.108685    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:43.213092 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:43.213053    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:47.437934 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:47.437896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:47.955735 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:47.955697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:49.288962 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.288886    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"]
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289251    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerName="util"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289270    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerName="util"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289282    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerName="extract"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289290    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerName="extract"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289302    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerName="pull"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289310    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerName="pull"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289322    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a024eea2-3868-406d-aadd-30673217b8f1" containerName="pull"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289330    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a024eea2-3868-406d-aadd-30673217b8f1" containerName="pull"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289339    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerName="pull"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289346    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerName="pull"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289359    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerName="pull"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289367    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerName="pull"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289380    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerName="util"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289388    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerName="util"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289399    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a024eea2-3868-406d-aadd-30673217b8f1" containerName="util"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289406    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a024eea2-3868-406d-aadd-30673217b8f1" containerName="util"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289413    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a024eea2-3868-406d-aadd-30673217b8f1" containerName="extract"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289421    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a024eea2-3868-406d-aadd-30673217b8f1" containerName="extract"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289429    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerName="extract"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289436    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerName="extract"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289444    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerName="extract"
Jul 21 20:46:49.289442 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289451    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerName="extract"
Jul 21 20:46:49.290217 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289469    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerName="util"
Jul 21 20:46:49.290217 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289475    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerName="util"
Jul 21 20:46:49.290217 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289523    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7d9fdeb-f64c-4b40-a579-5c588fc20786" containerName="extract"
Jul 21 20:46:49.290217 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289531    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4599dfc0-81db-4dee-9d05-0e6f4b91bdb6" containerName="extract"
Jul 21 20:46:49.290217 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289537    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e842868-4f6c-4dbb-882c-e913c9f7b20b" containerName="extract"
Jul 21 20:46:49.290217 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.289544    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="a024eea2-3868-406d-aadd-30673217b8f1" containerName="extract"
Jul 21 20:46:49.292609 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.292593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"
Jul 21 20:46:49.295423 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.295401    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xwq8g\""
Jul 21 20:46:49.295887 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.295874    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 20:46:49.296421 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.296403    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 21 20:46:49.296493 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.296439    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 20:46:49.303939 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.303916    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"]
Jul 21 20:46:49.349251 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.349218    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jpwk\" (UniqueName: \"kubernetes.io/projected/97d6b751-1106-4842-9122-bb1b5bec784d-kube-api-access-6jpwk\") pod \"dns-operator-controller-manager-864dd78c4c-w8ns6\" (UID: \"97d6b751-1106-4842-9122-bb1b5bec784d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"
Jul 21 20:46:49.450282 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.450230    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jpwk\" (UniqueName: \"kubernetes.io/projected/97d6b751-1106-4842-9122-bb1b5bec784d-kube-api-access-6jpwk\") pod \"dns-operator-controller-manager-864dd78c4c-w8ns6\" (UID: \"97d6b751-1106-4842-9122-bb1b5bec784d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"
Jul 21 20:46:49.462806 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.462774    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jpwk\" (UniqueName: \"kubernetes.io/projected/97d6b751-1106-4842-9122-bb1b5bec784d-kube-api-access-6jpwk\") pod \"dns-operator-controller-manager-864dd78c4c-w8ns6\" (UID: \"97d6b751-1106-4842-9122-bb1b5bec784d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"
Jul 21 20:46:49.603268 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.603178    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"
Jul 21 20:46:49.735604 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.735580    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"]
Jul 21 20:46:49.737766 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:46:49.737741    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d6b751_1106_4842_9122_bb1b5bec784d.slice/crio-0b1466726a7f5e8c44bc13342d300d662dd16cec3c71d9c83ca8b195eddd9f7f WatchSource:0}: Error finding container 0b1466726a7f5e8c44bc13342d300d662dd16cec3c71d9c83ca8b195eddd9f7f: Status 404 returned error can't find the container with id 0b1466726a7f5e8c44bc13342d300d662dd16cec3c71d9c83ca8b195eddd9f7f
Jul 21 20:46:49.993528 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:49.993493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:50.089682 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:50.089642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:50.094349 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:50.094324    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6" event={"ID":"97d6b751-1106-4842-9122-bb1b5bec784d","Type":"ContainerStarted","Data":"0b1466726a7f5e8c44bc13342d300d662dd16cec3c71d9c83ca8b195eddd9f7f"}
Jul 21 20:46:51.524239 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.524198    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"]
Jul 21 20:46:51.527719 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.527686    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:46:51.530868 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.530848    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-2zmzt\""
Jul 21 20:46:51.545526 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.545459    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"]
Jul 21 20:46:51.668004 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.667945    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv7vv\" (UniqueName: \"kubernetes.io/projected/1dacd878-1261-4df2-899b-d0dec9b14b66-kube-api-access-qv7vv\") pod \"limitador-operator-controller-manager-859d7b78bd-9qv2h\" (UID: \"1dacd878-1261-4df2-899b-d0dec9b14b66\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:46:51.768385 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.768347    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv7vv\" (UniqueName: \"kubernetes.io/projected/1dacd878-1261-4df2-899b-d0dec9b14b66-kube-api-access-qv7vv\") pod \"limitador-operator-controller-manager-859d7b78bd-9qv2h\" (UID: \"1dacd878-1261-4df2-899b-d0dec9b14b66\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:46:51.780115 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.780090    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv7vv\" (UniqueName: \"kubernetes.io/projected/1dacd878-1261-4df2-899b-d0dec9b14b66-kube-api-access-qv7vv\") pod \"limitador-operator-controller-manager-859d7b78bd-9qv2h\" (UID: \"1dacd878-1261-4df2-899b-d0dec9b14b66\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:46:51.840227 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.840206    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:46:51.948137 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.948096    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:51.983592 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:51.983568    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"]
Jul 21 20:46:51.985439 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:46:51.985413    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dacd878_1261_4df2_899b_d0dec9b14b66.slice/crio-1ac765f4a15d348021571d471c36974b93341779af70867ef739419bddbc8329 WatchSource:0}: Error finding container 1ac765f4a15d348021571d471c36974b93341779af70867ef739419bddbc8329: Status 404 returned error can't find the container with id 1ac765f4a15d348021571d471c36974b93341779af70867ef739419bddbc8329
Jul 21 20:46:52.056697 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:52.056612    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:52.107768 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:52.107731    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h" event={"ID":"1dacd878-1261-4df2-899b-d0dec9b14b66","Type":"ContainerStarted","Data":"1ac765f4a15d348021571d471c36974b93341779af70867ef739419bddbc8329"}
Jul 21 20:46:52.109300 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:52.109269    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6" event={"ID":"97d6b751-1106-4842-9122-bb1b5bec784d","Type":"ContainerStarted","Data":"d6d3032361cfc21f4bd9fdc71e67b8622979cd1c42bef0ea32c6341eb0b9c1ec"}
Jul 21 20:46:52.109429 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:52.109374    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"
Jul 21 20:46:52.145480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:52.145426    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6" podStartSLOduration=1.106147612 podStartE2EDuration="3.145403724s" totalImagesPullingTime="2.039256112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:46:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:46:52.144787056 +0000 UTC m=+439.354815488" watchObservedRunningTime="2026-07-21 20:46:52.145403724 +0000 UTC m=+439.355432147"
Jul 21 20:46:54.122156 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.122067    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h" event={"ID":"1dacd878-1261-4df2-899b-d0dec9b14b66","Type":"ContainerStarted","Data":"7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b"}
Jul 21 20:46:54.122509 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.122179    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:46:54.145383 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.145338    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h" podStartSLOduration=1.368769214 podStartE2EDuration="3.145323996s" totalImagesPullingTime="1.776554782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:46:54.142838545 +0000 UTC m=+441.352866979" watchObservedRunningTime="2026-07-21 20:46:54.145323996 +0000 UTC m=+441.355352419"
Jul 21 20:46:54.821153 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.821120    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"]
Jul 21 20:46:54.824415 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.824399    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:46:54.828480 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.828461    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5rr4s\""
Jul 21 20:46:54.846237 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.846213    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"]
Jul 21 20:46:54.894974 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.894945    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvxmt\" (UniqueName: \"kubernetes.io/projected/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-kube-api-access-hvxmt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jgxsz\" (UID: \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:46:54.895127 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.895003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jgxsz\" (UID: \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:46:54.996195 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.996159    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvxmt\" (UniqueName: \"kubernetes.io/projected/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-kube-api-access-hvxmt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jgxsz\" (UID: \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:46:54.996376 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.996210    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jgxsz\" (UID: \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:46:54.996535 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:54.996520    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jgxsz\" (UID: \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:46:55.006165 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:55.006137    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvxmt\" (UniqueName: \"kubernetes.io/projected/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-kube-api-access-hvxmt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jgxsz\" (UID: \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:46:55.134316 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:55.134244    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:46:55.270569 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:55.270543    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"]
Jul 21 20:46:55.272806 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:46:55.272778    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdaddb6a_cf2c_4a8f_87a6_cbf1a9e2a16f.slice/crio-3eb28199aa6e43158b4001281dea550df368beb1bcc54b74c8364b76c6631f71 WatchSource:0}: Error finding container 3eb28199aa6e43158b4001281dea550df368beb1bcc54b74c8364b76c6631f71: Status 404 returned error can't find the container with id 3eb28199aa6e43158b4001281dea550df368beb1bcc54b74c8364b76c6631f71
Jul 21 20:46:56.136698 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:56.136581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz" event={"ID":"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f","Type":"ContainerStarted","Data":"3eb28199aa6e43158b4001281dea550df368beb1bcc54b74c8364b76c6631f71"}
Jul 21 20:46:56.691009 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:56.690950    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:56.786770 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:56.786719    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:57.093257 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:57.093174    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:46:57.187301 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:46:57.187258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:00.956361 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:00.956319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:01.155829 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:01.155799    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:02.165770 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:02.165735    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz" event={"ID":"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f","Type":"ContainerStarted","Data":"f9a258308eb01929d79d6a7a913b6d972534411a0c9f2bbe87596577ff0e9dc3"}
Jul 21 20:47:02.166199 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:02.165813    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:47:02.197028 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:02.196967    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz" podStartSLOduration=2.346334325 podStartE2EDuration="8.196953148s" totalImagesPullingTime="5.850618823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:47:02.195556049 +0000 UTC m=+449.405584501" watchObservedRunningTime="2026-07-21 20:47:02.196953148 +0000 UTC m=+449.406981568"
Jul 21 20:47:03.118586 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:03.118554    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w8ns6"
Jul 21 20:47:03.927712 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:03.927674    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:04.051726 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:04.051678    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:04.155473 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:04.155439    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:05.129070 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:05.129038    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:47:07.454767 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:07.454731    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:07.554009 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:07.553957    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:13.173423 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:13.173392    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:47:14.841086 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.841050    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"]
Jul 21 20:47:14.845946 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.845923    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:14.865116 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.865089    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"]
Jul 21 20:47:14.865895 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.865867    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60107631-07aa-4165-bc96-ebd609728730-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" (UID: \"60107631-07aa-4165-bc96-ebd609728730\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:14.866033 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.865916    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg2z7\" (UniqueName: \"kubernetes.io/projected/60107631-07aa-4165-bc96-ebd609728730-kube-api-access-vg2z7\") pod \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" (UID: \"60107631-07aa-4165-bc96-ebd609728730\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:14.966419 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.966380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60107631-07aa-4165-bc96-ebd609728730-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" (UID: \"60107631-07aa-4165-bc96-ebd609728730\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:14.966583 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.966432    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg2z7\" (UniqueName: \"kubernetes.io/projected/60107631-07aa-4165-bc96-ebd609728730-kube-api-access-vg2z7\") pod \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" (UID: \"60107631-07aa-4165-bc96-ebd609728730\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:14.966827 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.966806    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60107631-07aa-4165-bc96-ebd609728730-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" (UID: \"60107631-07aa-4165-bc96-ebd609728730\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:14.977625 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:14.977599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg2z7\" (UniqueName: \"kubernetes.io/projected/60107631-07aa-4165-bc96-ebd609728730-kube-api-access-vg2z7\") pod \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" (UID: \"60107631-07aa-4165-bc96-ebd609728730\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:15.044164 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.044135    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"]
Jul 21 20:47:15.044391 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.044379    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:15.078101 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.078072    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"]
Jul 21 20:47:15.082849 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.082825    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"]
Jul 21 20:47:15.101058 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.100964    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"]
Jul 21 20:47:15.101247 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.101208    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz" podUID="bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" containerName="manager" containerID="cri-o://f9a258308eb01929d79d6a7a913b6d972534411a0c9f2bbe87596577ff0e9dc3" gracePeriod=2
Jul 21 20:47:15.120495 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.120472    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"]
Jul 21 20:47:15.141493 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.141468    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"]
Jul 21 20:47:15.141767 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.141756    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" containerName="manager"
Jul 21 20:47:15.141819 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.141769    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" containerName="manager"
Jul 21 20:47:15.141855 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.141821    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" containerName="manager"
Jul 21 20:47:15.144814 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.144796    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:15.147692 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.147669    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"]
Jul 21 20:47:15.147836 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.147816    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h" podUID="1dacd878-1261-4df2-899b-d0dec9b14b66" containerName="manager" containerID="cri-o://7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b" gracePeriod=2
Jul 21 20:47:15.165829 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.165799    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"]
Jul 21 20:47:15.168432 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.168410    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g6rn\" (UniqueName: \"kubernetes.io/projected/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-kube-api-access-9g6rn\") pod \"kuadrant-operator-controller-manager-86878c769f-gbdpp\" (UID: \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:15.168580 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.168442    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbdpp\" (UID: \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:15.173026 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.173004    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"]
Jul 21 20:47:15.173310 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.173297    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dacd878-1261-4df2-899b-d0dec9b14b66" containerName="manager"
Jul 21 20:47:15.173351 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.173313    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dacd878-1261-4df2-899b-d0dec9b14b66" containerName="manager"
Jul 21 20:47:15.173383 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.173377    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dacd878-1261-4df2-899b-d0dec9b14b66" containerName="manager"
Jul 21 20:47:15.177939 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.177922    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:15.187857 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.187833    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"]
Jul 21 20:47:15.206815 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.206789    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"]
Jul 21 20:47:15.220962 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:15.220906    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jgxsz\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" podUID="bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:47:15.230456 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.230432    2567 generic.go:356] "Generic (PLEG): container finished" podID="bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" containerID="f9a258308eb01929d79d6a7a913b6d972534411a0c9f2bbe87596577ff0e9dc3" exitCode=0
Jul 21 20:47:15.262196 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:15.262156    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-9qv2h\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" podUID="1dacd878-1261-4df2-899b-d0dec9b14b66" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:47:15.264733 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:15.264705    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jgxsz\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" podUID="bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:47:15.269214 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.269182    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6rn\" (UniqueName: \"kubernetes.io/projected/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-kube-api-access-9g6rn\") pod \"kuadrant-operator-controller-manager-86878c769f-gbdpp\" (UID: \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:15.269326 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.269223    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31c8d543-53a6-4b71-af34-11415fbff329-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6q2g6\" (UID: \"31c8d543-53a6-4b71-af34-11415fbff329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:15.269326 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.269257    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbdpp\" (UID: \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:15.269436 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.269323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rffdr\" (UniqueName: \"kubernetes.io/projected/31c8d543-53a6-4b71-af34-11415fbff329-kube-api-access-rffdr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6q2g6\" (UID: \"31c8d543-53a6-4b71-af34-11415fbff329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:15.269725 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.269697    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gbdpp\" (UID: \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:15.278693 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.278667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6rn\" (UniqueName: \"kubernetes.io/projected/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-kube-api-access-9g6rn\") pod \"kuadrant-operator-controller-manager-86878c769f-gbdpp\" (UID: \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:15.369823 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.369786    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31c8d543-53a6-4b71-af34-11415fbff329-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6q2g6\" (UID: \"31c8d543-53a6-4b71-af34-11415fbff329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:15.369935 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.369904    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rffdr\" (UniqueName: \"kubernetes.io/projected/31c8d543-53a6-4b71-af34-11415fbff329-kube-api-access-rffdr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6q2g6\" (UID: \"31c8d543-53a6-4b71-af34-11415fbff329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:15.370159 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.370141    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31c8d543-53a6-4b71-af34-11415fbff329-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6q2g6\" (UID: \"31c8d543-53a6-4b71-af34-11415fbff329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:15.375080 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.375058    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:47:15.380316 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.380294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rffdr\" (UniqueName: \"kubernetes.io/projected/31c8d543-53a6-4b71-af34-11415fbff329-kube-api-access-rffdr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6q2g6\" (UID: \"31c8d543-53a6-4b71-af34-11415fbff329\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:15.392067 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.392048    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:47:15.471166 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.471129    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvxmt\" (UniqueName: \"kubernetes.io/projected/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-kube-api-access-hvxmt\") pod \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\" (UID: \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\") "
Jul 21 20:47:15.471166 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.471166    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qv7vv\" (UniqueName: \"kubernetes.io/projected/1dacd878-1261-4df2-899b-d0dec9b14b66-kube-api-access-qv7vv\") pod \"1dacd878-1261-4df2-899b-d0dec9b14b66\" (UID: \"1dacd878-1261-4df2-899b-d0dec9b14b66\") "
Jul 21 20:47:15.471388 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.471250    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-extensions-socket-volume\") pod \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\" (UID: \"bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f\") "
Jul 21 20:47:15.471767 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.471741    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" (UID: "bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:47:15.473263 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.473238    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-kube-api-access-hvxmt" (OuterVolumeSpecName: "kube-api-access-hvxmt") pod "bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" (UID: "bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f"). InnerVolumeSpecName "kube-api-access-hvxmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:47:15.473263 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.473254    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dacd878-1261-4df2-899b-d0dec9b14b66-kube-api-access-qv7vv" (OuterVolumeSpecName: "kube-api-access-qv7vv") pod "1dacd878-1261-4df2-899b-d0dec9b14b66" (UID: "1dacd878-1261-4df2-899b-d0dec9b14b66"). InnerVolumeSpecName "kube-api-access-qv7vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:47:15.526202 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.526163    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:15.533104 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.533078    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:15.571933 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.571890    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvxmt\" (UniqueName: \"kubernetes.io/projected/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-kube-api-access-hvxmt\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:47:15.571933 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.571930    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv7vv\" (UniqueName: \"kubernetes.io/projected/1dacd878-1261-4df2-899b-d0dec9b14b66-kube-api-access-qv7vv\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:47:15.572187 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.571947    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f-extensions-socket-volume\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:47:15.686088 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.686057    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"]
Jul 21 20:47:15.690556 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:47:15.690526    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e1dfd9_2504_4379_a4e5_21aaa9ce2e7a.slice/crio-adc825f7fec28be8c8dc49febf58e387cc727433552b64c5b04d230504598571 WatchSource:0}: Error finding container adc825f7fec28be8c8dc49febf58e387cc727433552b64c5b04d230504598571: Status 404 returned error can't find the container with id adc825f7fec28be8c8dc49febf58e387cc727433552b64c5b04d230504598571
Jul 21 20:47:15.704507 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:15.704486    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"]
Jul 21 20:47:15.706682 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:47:15.706653    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c8d543_53a6_4b71_af34_11415fbff329.slice/crio-c1e93c67e990f7fad24b99e9c2cd5e4f79f246260059512c451a2c73f10748ba WatchSource:0}: Error finding container c1e93c67e990f7fad24b99e9c2cd5e4f79f246260059512c451a2c73f10748ba: Status 404 returned error can't find the container with id c1e93c67e990f7fad24b99e9c2cd5e4f79f246260059512c451a2c73f10748ba
Jul 21 20:47:16.040326 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:16.040277    2567 log.go:32] "RunPodSandbox from runtime service failed" err=<
Jul 21 20:47:16.040326 ip-10-0-130-177 kubenswrapper[2567]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-fdgc9_kuadrant-system_60107631-07aa-4165-bc96-ebd609728730_0(60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-fdgc9 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346" Netns:"/var/run/netns/83af4f87-7a4e-48d5-8996-58e1eaba3067" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-fdgc9;K8S_POD_INFRA_CONTAINER_ID=60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346;K8S_POD_UID=60107631-07aa-4165-bc96-ebd609728730" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9/60107631-07aa-4165-bc96-ebd609728730:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9 60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346 network default NAD default NAD key ] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9 60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346 network default NAD default NAD key ] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-55467d9f7-fdgc9" not found
Jul 21 20:47:16.040326 ip-10-0-130-177 kubenswrapper[2567]:         '
Jul 21 20:47:16.040326 ip-10-0-130-177 kubenswrapper[2567]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 21 20:47:16.040326 ip-10-0-130-177 kubenswrapper[2567]:  >
Jul 21 20:47:16.040888 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:16.040361    2567 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Jul 21 20:47:16.040888 ip-10-0-130-177 kubenswrapper[2567]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-fdgc9_kuadrant-system_60107631-07aa-4165-bc96-ebd609728730_0(60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-fdgc9 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346" Netns:"/var/run/netns/83af4f87-7a4e-48d5-8996-58e1eaba3067" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-fdgc9;K8S_POD_INFRA_CONTAINER_ID=60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346;K8S_POD_UID=60107631-07aa-4165-bc96-ebd609728730" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9/60107631-07aa-4165-bc96-ebd609728730:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9 60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346 network default NAD default NAD key ] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9 60d8a6312c68fb4645ae7eea92b77820ede1653a4154d2f8ffff6a7cf3787346 network default NAD default NAD key ] failed to get pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9: timed out waiting for pod after 1s: pods "kuadrant-operator-controller-manager-55467d9f7-fdgc9" not found
Jul 21 20:47:16.040888 ip-10-0-130-177 kubenswrapper[2567]:         '
Jul 21 20:47:16.040888 ip-10-0-130-177 kubenswrapper[2567]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 21 20:47:16.040888 ip-10-0-130-177 kubenswrapper[2567]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:16.244629 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.244538    2567 generic.go:356] "Generic (PLEG): container finished" podID="1dacd878-1261-4df2-899b-d0dec9b14b66" containerID="7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b" exitCode=0
Jul 21 20:47:16.244799 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.244699    2567 scope.go:117] "RemoveContainer" containerID="7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b"
Jul 21 20:47:16.244924 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.244891    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9qv2h"
Jul 21 20:47:16.246677 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.246648    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp" event={"ID":"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a","Type":"ContainerStarted","Data":"a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b"}
Jul 21 20:47:16.246771 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.246745    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:16.246826 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.246773    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp" event={"ID":"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a","Type":"ContainerStarted","Data":"adc825f7fec28be8c8dc49febf58e387cc727433552b64c5b04d230504598571"}
Jul 21 20:47:16.248451 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.248415    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6" event={"ID":"31c8d543-53a6-4b71-af34-11415fbff329","Type":"ContainerStarted","Data":"456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4"}
Jul 21 20:47:16.248565 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.248463    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6" event={"ID":"31c8d543-53a6-4b71-af34-11415fbff329","Type":"ContainerStarted","Data":"c1e93c67e990f7fad24b99e9c2cd5e4f79f246260059512c451a2c73f10748ba"}
Jul 21 20:47:16.248565 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.248507    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:16.249883 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.249866    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jgxsz"
Jul 21 20:47:16.249978 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.249884    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:16.255943 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.255924    2567 scope.go:117] "RemoveContainer" containerID="7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b"
Jul 21 20:47:16.256211 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:16.256194    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b\": container with ID starting with 7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b not found: ID does not exist" containerID="7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b"
Jul 21 20:47:16.256270 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.256218    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b"} err="failed to get container status \"7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b\": rpc error: code = NotFound desc = could not find container \"7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b\": container with ID starting with 7864996a4793592ae7b6449e571d9201711d937dcb779ae0b26bc451edf6f38b not found: ID does not exist"
Jul 21 20:47:16.256270 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.256248    2567 scope.go:117] "RemoveContainer" containerID="f9a258308eb01929d79d6a7a913b6d972534411a0c9f2bbe87596577ff0e9dc3"
Jul 21 20:47:16.258331 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.258308    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:16.271561 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.271520    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp" podStartSLOduration=1.27150632 podStartE2EDuration="1.27150632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:47:16.269340171 +0000 UTC m=+463.479368620" watchObservedRunningTime="2026-07-21 20:47:16.27150632 +0000 UTC m=+463.481534802"
Jul 21 20:47:16.271751 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:16.271728    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" podUID="60107631-07aa-4165-bc96-ebd609728730" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:16.277668 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.277649    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg2z7\" (UniqueName: \"kubernetes.io/projected/60107631-07aa-4165-bc96-ebd609728730-kube-api-access-vg2z7\") pod \"60107631-07aa-4165-bc96-ebd609728730\" (UID: \"60107631-07aa-4165-bc96-ebd609728730\") "
Jul 21 20:47:16.277763 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.277744    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60107631-07aa-4165-bc96-ebd609728730-extensions-socket-volume\") pod \"60107631-07aa-4165-bc96-ebd609728730\" (UID: \"60107631-07aa-4165-bc96-ebd609728730\") "
Jul 21 20:47:16.278020 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.277997    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60107631-07aa-4165-bc96-ebd609728730-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "60107631-07aa-4165-bc96-ebd609728730" (UID: "60107631-07aa-4165-bc96-ebd609728730"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:47:16.279697 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.279675    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60107631-07aa-4165-bc96-ebd609728730-kube-api-access-vg2z7" (OuterVolumeSpecName: "kube-api-access-vg2z7") pod "60107631-07aa-4165-bc96-ebd609728730" (UID: "60107631-07aa-4165-bc96-ebd609728730"). InnerVolumeSpecName "kube-api-access-vg2z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:47:16.293559 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.293515    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6" podStartSLOduration=1.293501822 podStartE2EDuration="1.293501822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:47:16.292287251 +0000 UTC m=+463.502315688" watchObservedRunningTime="2026-07-21 20:47:16.293501822 +0000 UTC m=+463.503530242"
Jul 21 20:47:16.378623 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.378587    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/60107631-07aa-4165-bc96-ebd609728730-extensions-socket-volume\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:47:16.378623 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:16.378618    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg2z7\" (UniqueName: \"kubernetes.io/projected/60107631-07aa-4165-bc96-ebd609728730-kube-api-access-vg2z7\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:47:17.257113 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:17.257080    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:17.259844 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:17.259818    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" podUID="60107631-07aa-4165-bc96-ebd609728730" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:17.265110 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:17.265084    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-fdgc9\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" podUID="60107631-07aa-4165-bc96-ebd609728730" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-fdgc9"
Jul 21 20:47:17.311219 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:17.311192    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dacd878-1261-4df2-899b-d0dec9b14b66" path="/var/lib/kubelet/pods/1dacd878-1261-4df2-899b-d0dec9b14b66/volumes"
Jul 21 20:47:17.311535 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:17.311521    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60107631-07aa-4165-bc96-ebd609728730" path="/var/lib/kubelet/pods/60107631-07aa-4165-bc96-ebd609728730/volumes"
Jul 21 20:47:17.311744 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:17.311733    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f" path="/var/lib/kubelet/pods/bdaddb6a-cf2c-4a8f-87a6-cbf1a9e2a16f/volumes"
Jul 21 20:47:27.259388 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.259355    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:27.259839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.259429    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:47:27.340422 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.340391    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"]
Jul 21 20:47:27.340613 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.340586    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6" podUID="31c8d543-53a6-4b71-af34-11415fbff329" containerName="manager" containerID="cri-o://456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4" gracePeriod=10
Jul 21 20:47:27.587089 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.587060    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:27.672204 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.672165    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rffdr\" (UniqueName: \"kubernetes.io/projected/31c8d543-53a6-4b71-af34-11415fbff329-kube-api-access-rffdr\") pod \"31c8d543-53a6-4b71-af34-11415fbff329\" (UID: \"31c8d543-53a6-4b71-af34-11415fbff329\") "
Jul 21 20:47:27.672398 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.672224    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31c8d543-53a6-4b71-af34-11415fbff329-extensions-socket-volume\") pod \"31c8d543-53a6-4b71-af34-11415fbff329\" (UID: \"31c8d543-53a6-4b71-af34-11415fbff329\") "
Jul 21 20:47:27.672670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.672638    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31c8d543-53a6-4b71-af34-11415fbff329-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "31c8d543-53a6-4b71-af34-11415fbff329" (UID: "31c8d543-53a6-4b71-af34-11415fbff329"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:47:27.674330 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.674305    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31c8d543-53a6-4b71-af34-11415fbff329-kube-api-access-rffdr" (OuterVolumeSpecName: "kube-api-access-rffdr") pod "31c8d543-53a6-4b71-af34-11415fbff329" (UID: "31c8d543-53a6-4b71-af34-11415fbff329"). InnerVolumeSpecName "kube-api-access-rffdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:47:27.773281 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.773248    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rffdr\" (UniqueName: \"kubernetes.io/projected/31c8d543-53a6-4b71-af34-11415fbff329-kube-api-access-rffdr\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:47:27.773281 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:27.773275    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/31c8d543-53a6-4b71-af34-11415fbff329-extensions-socket-volume\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:47:28.314496 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.314460    2567 generic.go:356] "Generic (PLEG): container finished" podID="31c8d543-53a6-4b71-af34-11415fbff329" containerID="456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4" exitCode=0
Jul 21 20:47:28.314887 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.314505    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"
Jul 21 20:47:28.314887 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.314536    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6" event={"ID":"31c8d543-53a6-4b71-af34-11415fbff329","Type":"ContainerDied","Data":"456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4"}
Jul 21 20:47:28.314887 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.314585    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6" event={"ID":"31c8d543-53a6-4b71-af34-11415fbff329","Type":"ContainerDied","Data":"c1e93c67e990f7fad24b99e9c2cd5e4f79f246260059512c451a2c73f10748ba"}
Jul 21 20:47:28.314887 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.314607    2567 scope.go:117] "RemoveContainer" containerID="456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4"
Jul 21 20:47:28.323562 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.323537    2567 scope.go:117] "RemoveContainer" containerID="456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4"
Jul 21 20:47:28.323806 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:47:28.323785    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4\": container with ID starting with 456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4 not found: ID does not exist" containerID="456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4"
Jul 21 20:47:28.323873 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.323814    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4"} err="failed to get container status \"456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4\": rpc error: code = NotFound desc = could not find container \"456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4\": container with ID starting with 456d8fa7e722d9745e367f6720c1962dac15b92fddbffee94fd3ce8b72693db4 not found: ID does not exist"
Jul 21 20:47:28.337242 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.337221    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"]
Jul 21 20:47:28.342795 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:28.342770    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6q2g6"]
Jul 21 20:47:29.310041 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:29.310009    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31c8d543-53a6-4b71-af34-11415fbff329" path="/var/lib/kubelet/pods/31c8d543-53a6-4b71-af34-11415fbff329/volumes"
Jul 21 20:47:33.232454 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:33.232425    2567 scope.go:117] "RemoveContainer" containerID="bd8e6b8bb653961a56b23f9837e59dc00ed7d6e17ca7a0279665e2dbf3dd6766"
Jul 21 20:47:43.717585 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.717548    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"]
Jul 21 20:47:43.717960 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.717944    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c8d543-53a6-4b71-af34-11415fbff329" containerName="manager"
Jul 21 20:47:43.718034 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.717963    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c8d543-53a6-4b71-af34-11415fbff329" containerName="manager"
Jul 21 20:47:43.718072 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.718041    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="31c8d543-53a6-4b71-af34-11415fbff329" containerName="manager"
Jul 21 20:47:43.722656 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.722637    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.724901 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.724883    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vk8v7\""
Jul 21 20:47:43.733799 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.733777    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"]
Jul 21 20:47:43.748948 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.748919    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:43.803847 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.803818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gd4t\" (UniqueName: \"kubernetes.io/projected/55009324-0e72-44b6-a2df-5652f7aaaaab-kube-api-access-6gd4t\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.804014 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.803868    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.804014 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.803945    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.804014 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.803979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.804148 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.804025    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.804148 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.804046    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.804148 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.804066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.804148 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.804088    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.804148 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.804139    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55009324-0e72-44b6-a2df-5652f7aaaaab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.891437 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.891400    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:43.905188 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905158    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905240    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905264    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905283    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905303    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905325 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905322    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55009324-0e72-44b6-a2df-5652f7aaaaab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905646 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905360    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gd4t\" (UniqueName: \"kubernetes.io/projected/55009324-0e72-44b6-a2df-5652f7aaaaab-kube-api-access-6gd4t\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905646 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905614    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905750 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905698    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905807 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905754    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.905927 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.905906    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.906124 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.906107    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55009324-0e72-44b6-a2df-5652f7aaaaab-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.907532 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.907506    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.907786 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.907768    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.913398 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.913380    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55009324-0e72-44b6-a2df-5652f7aaaaab-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:43.914212 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:43.914189    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gd4t\" (UniqueName: \"kubernetes.io/projected/55009324-0e72-44b6-a2df-5652f7aaaaab-kube-api-access-6gd4t\") pod \"maas-default-gateway-openshift-default-56654dcf7-bnqdr\" (UID: \"55009324-0e72-44b6-a2df-5652f7aaaaab\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:44.033048 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:44.032945    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:44.171333 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:44.171301    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"]
Jul 21 20:47:44.174561 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:47:44.174531    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55009324_0e72_44b6_a2df_5652f7aaaaab.slice/crio-7b12cb5c6c70a9eab1856aa6ca0202cb85688f2d499fbe1cc9de29065ceb8798 WatchSource:0}: Error finding container 7b12cb5c6c70a9eab1856aa6ca0202cb85688f2d499fbe1cc9de29065ceb8798: Status 404 returned error can't find the container with id 7b12cb5c6c70a9eab1856aa6ca0202cb85688f2d499fbe1cc9de29065ceb8798
Jul 21 20:47:44.389392 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:44.389305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" event={"ID":"55009324-0e72-44b6-a2df-5652f7aaaaab","Type":"ContainerStarted","Data":"7b12cb5c6c70a9eab1856aa6ca0202cb85688f2d499fbe1cc9de29065ceb8798"}
Jul 21 20:47:46.742516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:46.742479    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 20:47:46.742801 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:46.742548    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 20:47:46.742801 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:46.742577    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 20:47:47.408452 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:47.408414    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" event={"ID":"55009324-0e72-44b6-a2df-5652f7aaaaab","Type":"ContainerStarted","Data":"cce3d9625175a3d4d5f0c85a51c34d97679c0b582e658068730737fb6ff3d046"}
Jul 21 20:47:47.431837 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:47.431788    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" podStartSLOduration=1.868769487 podStartE2EDuration="4.431771513s" totalImagesPullingTime="2.563002026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:47:47.429781739 +0000 UTC m=+494.639810181" watchObservedRunningTime="2026-07-21 20:47:47.431771513 +0000 UTC m=+494.641799933"
Jul 21 20:47:48.033911 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.033871    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:48.038962 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.038933    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:48.413892 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.413866    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:48.414823 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.414802    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:47:48.450606 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.450572    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"]
Jul 21 20:47:48.451787 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.451756    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:48.454676 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.454654    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:48.457345 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.457310    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tlcgm\""
Jul 21 20:47:48.457345 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.457318    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 21 20:47:48.466180 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.466047    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"]
Jul 21 20:47:48.544802 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.544767    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"]
Jul 21 20:47:48.545176 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.545150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw8c9\" (UniqueName: \"kubernetes.io/projected/1634c946-bb92-42f8-bf6b-8bbeb771479b-kube-api-access-bw8c9\") pod \"limitador-limitador-64c94fd64f-kg9bx\" (UID: \"1634c946-bb92-42f8-bf6b-8bbeb771479b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:48.545343 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.545266    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1634c946-bb92-42f8-bf6b-8bbeb771479b-config-file\") pod \"limitador-limitador-64c94fd64f-kg9bx\" (UID: \"1634c946-bb92-42f8-bf6b-8bbeb771479b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:48.559425 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.559393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:48.646102 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.646065    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw8c9\" (UniqueName: \"kubernetes.io/projected/1634c946-bb92-42f8-bf6b-8bbeb771479b-kube-api-access-bw8c9\") pod \"limitador-limitador-64c94fd64f-kg9bx\" (UID: \"1634c946-bb92-42f8-bf6b-8bbeb771479b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:48.646353 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.646130    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1634c946-bb92-42f8-bf6b-8bbeb771479b-config-file\") pod \"limitador-limitador-64c94fd64f-kg9bx\" (UID: \"1634c946-bb92-42f8-bf6b-8bbeb771479b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:48.646779 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.646752    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1634c946-bb92-42f8-bf6b-8bbeb771479b-config-file\") pod \"limitador-limitador-64c94fd64f-kg9bx\" (UID: \"1634c946-bb92-42f8-bf6b-8bbeb771479b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:48.654744 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.654718    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw8c9\" (UniqueName: \"kubernetes.io/projected/1634c946-bb92-42f8-bf6b-8bbeb771479b-kube-api-access-bw8c9\") pod \"limitador-limitador-64c94fd64f-kg9bx\" (UID: \"1634c946-bb92-42f8-bf6b-8bbeb771479b\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:48.659536 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.659502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:48.768883 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.768805    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:48.905012 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:48.904970    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"]
Jul 21 20:47:48.907626 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:47:48.907593    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1634c946_bb92_42f8_bf6b_8bbeb771479b.slice/crio-aa98c7f6a3d138195fd708d4c83d71bce0ec5c420eb5671a05a4a34276ae9d59 WatchSource:0}: Error finding container aa98c7f6a3d138195fd708d4c83d71bce0ec5c420eb5671a05a4a34276ae9d59: Status 404 returned error can't find the container with id aa98c7f6a3d138195fd708d4c83d71bce0ec5c420eb5671a05a4a34276ae9d59
Jul 21 20:47:49.300975 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:49.300920    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:49.384306 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:49.384275    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:47:49.421347 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:49.421308    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx" event={"ID":"1634c946-bb92-42f8-bf6b-8bbeb771479b","Type":"ContainerStarted","Data":"aa98c7f6a3d138195fd708d4c83d71bce0ec5c420eb5671a05a4a34276ae9d59"}
Jul 21 20:47:52.439651 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:52.439617    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx" event={"ID":"1634c946-bb92-42f8-bf6b-8bbeb771479b","Type":"ContainerStarted","Data":"7344fedf5d400eefeebbd066c826dfe636e34456389b2e191b263e4516830056"}
Jul 21 20:47:52.440070 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:52.439695    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:47:52.459441 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:52.459392    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx" podStartSLOduration=1.7490334920000001 podStartE2EDuration="4.459378305s" totalImagesPullingTime="2.710344813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:47:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:47:52.458397617 +0000 UTC m=+499.668426065" watchObservedRunningTime="2026-07-21 20:47:52.459378305 +0000 UTC m=+499.669406726"
Jul 21 20:47:54.456131 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:47:54.456096    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:03.445584 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:03.445554    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:48:03.911260 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:03.911166    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"]
Jul 21 20:48:03.911428 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:03.911351    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx" podUID="1634c946-bb92-42f8-bf6b-8bbeb771479b" containerName="limitador" containerID="cri-o://7344fedf5d400eefeebbd066c826dfe636e34456389b2e191b263e4516830056" gracePeriod=30
Jul 21 20:48:04.495178 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:04.495097    2567 generic.go:356] "Generic (PLEG): container finished" podID="1634c946-bb92-42f8-bf6b-8bbeb771479b" containerID="7344fedf5d400eefeebbd066c826dfe636e34456389b2e191b263e4516830056" exitCode=0
Jul 21 20:48:04.495178 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:04.495130    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx" event={"ID":"1634c946-bb92-42f8-bf6b-8bbeb771479b","Type":"ContainerDied","Data":"7344fedf5d400eefeebbd066c826dfe636e34456389b2e191b263e4516830056"}
Jul 21 20:48:04.846575 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:04.846548    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:48:04.980746 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:04.980707    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1634c946-bb92-42f8-bf6b-8bbeb771479b-config-file\") pod \"1634c946-bb92-42f8-bf6b-8bbeb771479b\" (UID: \"1634c946-bb92-42f8-bf6b-8bbeb771479b\") "
Jul 21 20:48:04.981073 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:04.981053    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bw8c9\" (UniqueName: \"kubernetes.io/projected/1634c946-bb92-42f8-bf6b-8bbeb771479b-kube-api-access-bw8c9\") pod \"1634c946-bb92-42f8-bf6b-8bbeb771479b\" (UID: \"1634c946-bb92-42f8-bf6b-8bbeb771479b\") "
Jul 21 20:48:04.981597 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:04.981182    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1634c946-bb92-42f8-bf6b-8bbeb771479b-config-file" (OuterVolumeSpecName: "config-file") pod "1634c946-bb92-42f8-bf6b-8bbeb771479b" (UID: "1634c946-bb92-42f8-bf6b-8bbeb771479b"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 20:48:04.981927 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:04.981910    2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1634c946-bb92-42f8-bf6b-8bbeb771479b-config-file\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:48:04.984258 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:04.984224    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1634c946-bb92-42f8-bf6b-8bbeb771479b-kube-api-access-bw8c9" (OuterVolumeSpecName: "kube-api-access-bw8c9") pod "1634c946-bb92-42f8-bf6b-8bbeb771479b" (UID: "1634c946-bb92-42f8-bf6b-8bbeb771479b"). InnerVolumeSpecName "kube-api-access-bw8c9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:48:05.082845 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:05.082767    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw8c9\" (UniqueName: \"kubernetes.io/projected/1634c946-bb92-42f8-bf6b-8bbeb771479b-kube-api-access-bw8c9\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:48:05.507857 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:05.507813    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx" event={"ID":"1634c946-bb92-42f8-bf6b-8bbeb771479b","Type":"ContainerDied","Data":"aa98c7f6a3d138195fd708d4c83d71bce0ec5c420eb5671a05a4a34276ae9d59"}
Jul 21 20:48:05.508358 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:05.507912    2567 scope.go:117] "RemoveContainer" containerID="7344fedf5d400eefeebbd066c826dfe636e34456389b2e191b263e4516830056"
Jul 21 20:48:05.508358 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:05.507838    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"
Jul 21 20:48:05.525476 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:05.525454    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"]
Jul 21 20:48:05.529179 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:05.529155    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-kg9bx"]
Jul 21 20:48:07.289910 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:07.289871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:07.311354 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:07.311321    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1634c946-bb92-42f8-bf6b-8bbeb771479b" path="/var/lib/kubelet/pods/1634c946-bb92-42f8-bf6b-8bbeb771479b/volumes"
Jul 21 20:48:07.394570 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:07.394536    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:09.783946 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:09.783909    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:09.898356 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:09.898319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:10.246235 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.246200    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"]
Jul 21 20:48:10.246517 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.246505    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1634c946-bb92-42f8-bf6b-8bbeb771479b" containerName="limitador"
Jul 21 20:48:10.246568 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.246518    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1634c946-bb92-42f8-bf6b-8bbeb771479b" containerName="limitador"
Jul 21 20:48:10.246605 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.246577    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1634c946-bb92-42f8-bf6b-8bbeb771479b" containerName="limitador"
Jul 21 20:48:10.250840 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.250821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:10.257019 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.256972    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 20:48:10.257160 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.257055    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 21 20:48:10.257775 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.257751    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 20:48:10.257946 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.257757    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-7wktc\""
Jul 21 20:48:10.260532 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.260508    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"]
Jul 21 20:48:10.278955 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.278909    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:10.430432 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.430393    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/48adffdd-ae68-4b19-bccb-351ce95b13ac-data\") pod \"postgres-69fb4594cf-mkhxt\" (UID: \"48adffdd-ae68-4b19-bccb-351ce95b13ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:10.430604 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.430444    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrh2q\" (UniqueName: \"kubernetes.io/projected/48adffdd-ae68-4b19-bccb-351ce95b13ac-kube-api-access-wrh2q\") pod \"postgres-69fb4594cf-mkhxt\" (UID: \"48adffdd-ae68-4b19-bccb-351ce95b13ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:10.531032 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.530923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/48adffdd-ae68-4b19-bccb-351ce95b13ac-data\") pod \"postgres-69fb4594cf-mkhxt\" (UID: \"48adffdd-ae68-4b19-bccb-351ce95b13ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:10.531032 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.531006    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrh2q\" (UniqueName: \"kubernetes.io/projected/48adffdd-ae68-4b19-bccb-351ce95b13ac-kube-api-access-wrh2q\") pod \"postgres-69fb4594cf-mkhxt\" (UID: \"48adffdd-ae68-4b19-bccb-351ce95b13ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:10.531337 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.531315    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/48adffdd-ae68-4b19-bccb-351ce95b13ac-data\") pod \"postgres-69fb4594cf-mkhxt\" (UID: \"48adffdd-ae68-4b19-bccb-351ce95b13ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:10.539909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.539887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrh2q\" (UniqueName: \"kubernetes.io/projected/48adffdd-ae68-4b19-bccb-351ce95b13ac-kube-api-access-wrh2q\") pod \"postgres-69fb4594cf-mkhxt\" (UID: \"48adffdd-ae68-4b19-bccb-351ce95b13ac\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:10.566720 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.566692    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:10.695280 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:10.695231    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"]
Jul 21 20:48:10.698175 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:48:10.698142    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48adffdd_ae68_4b19_bccb_351ce95b13ac.slice/crio-74a0137d441e3c64c89e17ed59f87692754b3da78dc7b44220bcd0ce5af358ec WatchSource:0}: Error finding container 74a0137d441e3c64c89e17ed59f87692754b3da78dc7b44220bcd0ce5af358ec: Status 404 returned error can't find the container with id 74a0137d441e3c64c89e17ed59f87692754b3da78dc7b44220bcd0ce5af358ec
Jul 21 20:48:11.541549 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:11.541460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt" event={"ID":"48adffdd-ae68-4b19-bccb-351ce95b13ac","Type":"ContainerStarted","Data":"74a0137d441e3c64c89e17ed59f87692754b3da78dc7b44220bcd0ce5af358ec"}
Jul 21 20:48:16.570472 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:16.570429    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt" event={"ID":"48adffdd-ae68-4b19-bccb-351ce95b13ac","Type":"ContainerStarted","Data":"c9f23ef1bebdf1015869725bcaa0a8231b57a513c86fcc1d21c6021966f4149c"}
Jul 21 20:48:16.570937 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:16.570558    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:16.586809 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:16.586748    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt" podStartSLOduration=1.754648496 podStartE2EDuration="6.586729041s" totalImagesPullingTime="4.832080545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:48:16.586268047 +0000 UTC m=+523.796296506" watchObservedRunningTime="2026-07-21 20:48:16.586729041 +0000 UTC m=+523.796757464"
Jul 21 20:48:17.983377 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:17.983337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:18.056201 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:18.056169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:18.147917 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:18.147883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:18.247581 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:18.247496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:18.348808 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:18.348766    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:22.604054 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:22.604029    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mkhxt"
Jul 21 20:48:22.942104 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:22.942066    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:23.161820 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:23.161786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:29.710747 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.710690    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:29.720501 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.720474    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dd584c847-bxwzp"]
Jul 21 20:48:29.723818 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.723799    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:29.726696 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.726676    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8xkcb\""
Jul 21 20:48:29.727478 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.727463    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 20:48:29.727700 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.727683    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 20:48:29.738186 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.738160    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dd584c847-bxwzp"]
Jul 21 20:48:29.781571 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.781538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrbrr\" (UniqueName: \"kubernetes.io/projected/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-kube-api-access-vrbrr\") pod \"maas-controller-7dd584c847-bxwzp\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") " pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:29.781725 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.781579    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert\") pod \"maas-controller-7dd584c847-bxwzp\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") " pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:29.808333 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.808302    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:29.882608 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.882573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrbrr\" (UniqueName: \"kubernetes.io/projected/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-kube-api-access-vrbrr\") pod \"maas-controller-7dd584c847-bxwzp\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") " pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:29.882750 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.882619    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert\") pod \"maas-controller-7dd584c847-bxwzp\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") " pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:29.882750 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:48:29.882742    2567 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 21 20:48:29.882822 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:48:29.882806    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert podName:307b6d26-e865-4e9b-ab87-9ac9d21a6b60 nodeName:}" failed. No retries permitted until 2026-07-21 20:48:30.382785827 +0000 UTC m=+537.592814226 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert") pod "maas-controller-7dd584c847-bxwzp" (UID: "307b6d26-e865-4e9b-ab87-9ac9d21a6b60") : secret "maas-controller-webhook-cert" not found
Jul 21 20:48:29.891874 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.891847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrbrr\" (UniqueName: \"kubernetes.io/projected/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-kube-api-access-vrbrr\") pod \"maas-controller-7dd584c847-bxwzp\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") " pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:29.911456 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.911419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:29.920904 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.920878    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-4qtwl"]
Jul 21 20:48:29.924050 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.924035    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:29.942449 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.942426    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-4qtwl"]
Jul 21 20:48:29.983079 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.983005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert\") pod \"maas-controller-84f9cc6f45-4qtwl\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") " pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:29.983079 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:29.983062    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqhqx\" (UniqueName: \"kubernetes.io/projected/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-kube-api-access-jqhqx\") pod \"maas-controller-84f9cc6f45-4qtwl\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") " pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:30.084467 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.084431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqhqx\" (UniqueName: \"kubernetes.io/projected/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-kube-api-access-jqhqx\") pod \"maas-controller-84f9cc6f45-4qtwl\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") " pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:30.084627 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.084513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert\") pod \"maas-controller-84f9cc6f45-4qtwl\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") " pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:30.084667 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:48:30.084618    2567 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 21 20:48:30.084701 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:48:30.084695    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert podName:883e7ff5-8efb-49f8-8fe4-eabca37dbb10 nodeName:}" failed. No retries permitted until 2026-07-21 20:48:30.584681861 +0000 UTC m=+537.794710260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert") pod "maas-controller-84f9cc6f45-4qtwl" (UID: "883e7ff5-8efb-49f8-8fe4-eabca37dbb10") : secret "maas-controller-webhook-cert" not found
Jul 21 20:48:30.096159 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.096127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqhqx\" (UniqueName: \"kubernetes.io/projected/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-kube-api-access-jqhqx\") pod \"maas-controller-84f9cc6f45-4qtwl\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") " pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:30.188922 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.188885    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:30.387433 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.387348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert\") pod \"maas-controller-7dd584c847-bxwzp\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") " pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:30.389665 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.389637    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert\") pod \"maas-controller-7dd584c847-bxwzp\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") " pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:30.588511 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.588476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert\") pod \"maas-controller-84f9cc6f45-4qtwl\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") " pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:30.590807 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.590789    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert\") pod \"maas-controller-84f9cc6f45-4qtwl\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") " pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:30.634126 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.634098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:30.766284 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.766229    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dd584c847-bxwzp"]
Jul 21 20:48:30.768590 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:48:30.768563    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307b6d26_e865_4e9b_ab87_9ac9d21a6b60.slice/crio-7ba4fde3566c3625bd75628f469a2df567bb63e3a1ec347cbab9f3230e00ef52 WatchSource:0}: Error finding container 7ba4fde3566c3625bd75628f469a2df567bb63e3a1ec347cbab9f3230e00ef52: Status 404 returned error can't find the container with id 7ba4fde3566c3625bd75628f469a2df567bb63e3a1ec347cbab9f3230e00ef52
Jul 21 20:48:30.834606 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.834574    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:30.963251 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:30.963228    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-4qtwl"]
Jul 21 20:48:30.965233 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:48:30.965207    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod883e7ff5_8efb_49f8_8fe4_eabca37dbb10.slice/crio-2d5f0cf3da431de33de9a0f6f60d79267824dcf0f54e5a6b6072c319236b9965 WatchSource:0}: Error finding container 2d5f0cf3da431de33de9a0f6f60d79267824dcf0f54e5a6b6072c319236b9965: Status 404 returned error can't find the container with id 2d5f0cf3da431de33de9a0f6f60d79267824dcf0f54e5a6b6072c319236b9965
Jul 21 20:48:31.653976 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:31.653714    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dd584c847-bxwzp" event={"ID":"307b6d26-e865-4e9b-ab87-9ac9d21a6b60","Type":"ContainerStarted","Data":"7ba4fde3566c3625bd75628f469a2df567bb63e3a1ec347cbab9f3230e00ef52"}
Jul 21 20:48:31.655607 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:31.655577    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl" event={"ID":"883e7ff5-8efb-49f8-8fe4-eabca37dbb10","Type":"ContainerStarted","Data":"2d5f0cf3da431de33de9a0f6f60d79267824dcf0f54e5a6b6072c319236b9965"}
Jul 21 20:48:33.315388 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.315363    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 20:48:33.544221 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.544137    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:33.644280 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.644237    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:48:33.668136 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.668098    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl" event={"ID":"883e7ff5-8efb-49f8-8fe4-eabca37dbb10","Type":"ContainerStarted","Data":"b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3"}
Jul 21 20:48:33.668288 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.668167    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:33.669498 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.669470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dd584c847-bxwzp" event={"ID":"307b6d26-e865-4e9b-ab87-9ac9d21a6b60","Type":"ContainerStarted","Data":"7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253"}
Jul 21 20:48:33.669619 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.669600    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:33.700413 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.700356    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl" podStartSLOduration=2.347801166 podStartE2EDuration="4.700338357s" totalImagesPullingTime="2.352537191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:48:33.697927566 +0000 UTC m=+540.907955986" watchObservedRunningTime="2026-07-21 20:48:33.700338357 +0000 UTC m=+540.910366779"
Jul 21 20:48:33.726617 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:33.726560    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dd584c847-bxwzp" podStartSLOduration=2.183941707 podStartE2EDuration="4.726545428s" totalImagesPullingTime="2.542603721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:48:33.725098373 +0000 UTC m=+540.935126829" watchObservedRunningTime="2026-07-21 20:48:33.726545428 +0000 UTC m=+540.936573848"
Jul 21 20:48:44.676941 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:44.676912    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:48:44.677508 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:44.677008    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:44.741304 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:44.741268    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dd584c847-bxwzp"]
Jul 21 20:48:44.741573 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:44.741515    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7dd584c847-bxwzp" podUID="307b6d26-e865-4e9b-ab87-9ac9d21a6b60" containerName="manager" containerID="cri-o://7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253" gracePeriod=30
Jul 21 20:48:44.982643 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:44.982616    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:45.122126 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.122098    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert\") pod \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") "
Jul 21 20:48:45.122281 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.122185    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrbrr\" (UniqueName: \"kubernetes.io/projected/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-kube-api-access-vrbrr\") pod \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\" (UID: \"307b6d26-e865-4e9b-ab87-9ac9d21a6b60\") "
Jul 21 20:48:45.124215 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.124183    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "307b6d26-e865-4e9b-ab87-9ac9d21a6b60" (UID: "307b6d26-e865-4e9b-ab87-9ac9d21a6b60"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 20:48:45.124319 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.124247    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-kube-api-access-vrbrr" (OuterVolumeSpecName: "kube-api-access-vrbrr") pod "307b6d26-e865-4e9b-ab87-9ac9d21a6b60" (UID: "307b6d26-e865-4e9b-ab87-9ac9d21a6b60"). InnerVolumeSpecName "kube-api-access-vrbrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:48:45.222889 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.222801    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-webhook-cert\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:48:45.222889 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.222833    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrbrr\" (UniqueName: \"kubernetes.io/projected/307b6d26-e865-4e9b-ab87-9ac9d21a6b60-kube-api-access-vrbrr\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:48:45.732364 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.732327    2567 generic.go:356] "Generic (PLEG): container finished" podID="307b6d26-e865-4e9b-ab87-9ac9d21a6b60" containerID="7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253" exitCode=0
Jul 21 20:48:45.732777 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.732374    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dd584c847-bxwzp"
Jul 21 20:48:45.732777 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.732384    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dd584c847-bxwzp" event={"ID":"307b6d26-e865-4e9b-ab87-9ac9d21a6b60","Type":"ContainerDied","Data":"7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253"}
Jul 21 20:48:45.732777 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.732452    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dd584c847-bxwzp" event={"ID":"307b6d26-e865-4e9b-ab87-9ac9d21a6b60","Type":"ContainerDied","Data":"7ba4fde3566c3625bd75628f469a2df567bb63e3a1ec347cbab9f3230e00ef52"}
Jul 21 20:48:45.732777 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.732467    2567 scope.go:117] "RemoveContainer" containerID="7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253"
Jul 21 20:48:45.741008 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.740970    2567 scope.go:117] "RemoveContainer" containerID="7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253"
Jul 21 20:48:45.741259 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:48:45.741236    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253\": container with ID starting with 7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253 not found: ID does not exist" containerID="7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253"
Jul 21 20:48:45.741305 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.741268    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253"} err="failed to get container status \"7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253\": rpc error: code = NotFound desc = could not find container \"7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253\": container with ID starting with 7e9bee2befc8da4d680a8e0f2f8af68c3a29fc0f311724150d823183331dc253 not found: ID does not exist"
Jul 21 20:48:45.754738 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.754707    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dd584c847-bxwzp"]
Jul 21 20:48:45.764654 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:45.764630    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7dd584c847-bxwzp"]
Jul 21 20:48:47.310446 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:48:47.310407    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="307b6d26-e865-4e9b-ab87-9ac9d21a6b60" path="/var/lib/kubelet/pods/307b6d26-e865-4e9b-ab87-9ac9d21a6b60/volumes"
Jul 21 20:49:05.286431 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.286397    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:05.387114 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.387078    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:05.769280 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.769238    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:05.814566 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.814532    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"]
Jul 21 20:49:05.814861 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.814849    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="307b6d26-e865-4e9b-ab87-9ac9d21a6b60" containerName="manager"
Jul 21 20:49:05.814907 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.814863    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="307b6d26-e865-4e9b-ab87-9ac9d21a6b60" containerName="manager"
Jul 21 20:49:05.814942 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.814927    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="307b6d26-e865-4e9b-ab87-9ac9d21a6b60" containerName="manager"
Jul 21 20:49:05.823047 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.823019    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:05.825554 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.825530    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 21 20:49:05.833308 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.833282    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"]
Jul 21 20:49:05.897516 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.897468    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b9d3a7-de1e-4fc0-9e33-165e22403414-openshift-service-ca\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:05.897676 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.897585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5jq4\" (UniqueName: \"kubernetes.io/projected/49b9d3a7-de1e-4fc0-9e33-165e22403414-kube-api-access-h5jq4\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:05.897676 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.897615    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49b9d3a7-de1e-4fc0-9e33-165e22403414-maas-api-tls\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:05.998046 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.998012    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:05.998395 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.998209    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b9d3a7-de1e-4fc0-9e33-165e22403414-openshift-service-ca\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:05.998684 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.998661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5jq4\" (UniqueName: \"kubernetes.io/projected/49b9d3a7-de1e-4fc0-9e33-165e22403414-kube-api-access-h5jq4\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:05.998857 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.998807    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49b9d3a7-de1e-4fc0-9e33-165e22403414-maas-api-tls\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:05.999847 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:05.999824    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49b9d3a7-de1e-4fc0-9e33-165e22403414-openshift-service-ca\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:06.002037 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.002013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49b9d3a7-de1e-4fc0-9e33-165e22403414-maas-api-tls\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:06.008958 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.008933    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5jq4\" (UniqueName: \"kubernetes.io/projected/49b9d3a7-de1e-4fc0-9e33-165e22403414-kube-api-access-h5jq4\") pod \"maas-api-6f45cb59b7-pmg28\" (UID: \"49b9d3a7-de1e-4fc0-9e33-165e22403414\") " pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:06.134748 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.134673    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:06.284264 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.284237    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"]
Jul 21 20:49:06.286732 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:49:06.286691    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b9d3a7_de1e_4fc0_9e33_165e22403414.slice/crio-692f8e16ec736cd35454479a9058920a99af410b3b251b2cb20c9f73f92584b6 WatchSource:0}: Error finding container 692f8e16ec736cd35454479a9058920a99af410b3b251b2cb20c9f73f92584b6: Status 404 returned error can't find the container with id 692f8e16ec736cd35454479a9058920a99af410b3b251b2cb20c9f73f92584b6
Jul 21 20:49:06.441572 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.441535    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:06.588834 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.588797    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:06.837301 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.837210    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28" event={"ID":"49b9d3a7-de1e-4fc0-9e33-165e22403414","Type":"ContainerStarted","Data":"692f8e16ec736cd35454479a9058920a99af410b3b251b2cb20c9f73f92584b6"}
Jul 21 20:49:06.860268 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.860231    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-b48b6"]
Jul 21 20:49:06.865648 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.865624    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:06.871309 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.871271    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 21 20:49:06.871827 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.871683    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-t89sk\""
Jul 21 20:49:06.893697 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.893667    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-b48b6"]
Jul 21 20:49:06.907304 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.907264    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58bae39a-f148-4e65-b2ae-80093e13b586-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-b48b6\" (UID: \"58bae39a-f148-4e65-b2ae-80093e13b586\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:06.907461 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:06.907352    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fnvp\" (UniqueName: \"kubernetes.io/projected/58bae39a-f148-4e65-b2ae-80093e13b586-kube-api-access-9fnvp\") pod \"payload-pre-processing-7677b774bb-b48b6\" (UID: \"58bae39a-f148-4e65-b2ae-80093e13b586\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:07.008122 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:07.008085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fnvp\" (UniqueName: \"kubernetes.io/projected/58bae39a-f148-4e65-b2ae-80093e13b586-kube-api-access-9fnvp\") pod \"payload-pre-processing-7677b774bb-b48b6\" (UID: \"58bae39a-f148-4e65-b2ae-80093e13b586\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:07.008328 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:07.008190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58bae39a-f148-4e65-b2ae-80093e13b586-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-b48b6\" (UID: \"58bae39a-f148-4e65-b2ae-80093e13b586\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:07.009598 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:07.009569    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58bae39a-f148-4e65-b2ae-80093e13b586-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-b48b6\" (UID: \"58bae39a-f148-4e65-b2ae-80093e13b586\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:07.020645 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:07.020613    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fnvp\" (UniqueName: \"kubernetes.io/projected/58bae39a-f148-4e65-b2ae-80093e13b586-kube-api-access-9fnvp\") pod \"payload-pre-processing-7677b774bb-b48b6\" (UID: \"58bae39a-f148-4e65-b2ae-80093e13b586\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:07.176100 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:07.176066    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:07.340729 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:07.340692    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-b48b6"]
Jul 21 20:49:07.343962 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:49:07.343923    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58bae39a_f148_4e65_b2ae_80093e13b586.slice/crio-5937b7da61e6b9c0b54516b9fc2cef43901b7e29a18cbc60ec326efcd9a457ed WatchSource:0}: Error finding container 5937b7da61e6b9c0b54516b9fc2cef43901b7e29a18cbc60ec326efcd9a457ed: Status 404 returned error can't find the container with id 5937b7da61e6b9c0b54516b9fc2cef43901b7e29a18cbc60ec326efcd9a457ed
Jul 21 20:49:07.872367 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:07.872292    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6" event={"ID":"58bae39a-f148-4e65-b2ae-80093e13b586","Type":"ContainerStarted","Data":"5937b7da61e6b9c0b54516b9fc2cef43901b7e29a18cbc60ec326efcd9a457ed"}
Jul 21 20:49:08.881268 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:08.881221    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28" event={"ID":"49b9d3a7-de1e-4fc0-9e33-165e22403414","Type":"ContainerStarted","Data":"f6fe8ae0d94e43353f313434ffc4f2948fc63722bb95fb92ddbbcd4ee391c79c"}
Jul 21 20:49:08.881743 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:08.881516    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:08.901915 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:08.901850    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28" podStartSLOduration=2.004953802 podStartE2EDuration="3.901826923s" totalImagesPullingTime="1.896873121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:49:08.8989737 +0000 UTC m=+576.109002157" watchObservedRunningTime="2026-07-21 20:49:08.901826923 +0000 UTC m=+576.111855343"
Jul 21 20:49:09.888154 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:09.888065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6" event={"ID":"58bae39a-f148-4e65-b2ae-80093e13b586","Type":"ContainerStarted","Data":"95ecf7e6c34e1b5ce31644bd72d42dfdd0c74a9bb5429a85cdb3d0b5f93664a2"}
Jul 21 20:49:09.888500 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:09.888386    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:09.908847 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:09.908791    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6" podStartSLOduration=1.65076556 podStartE2EDuration="3.908773091s" totalImagesPullingTime="2.258007531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:49:09.907567828 +0000 UTC m=+577.117596264" watchObservedRunningTime="2026-07-21 20:49:09.908773091 +0000 UTC m=+577.118801511"
Jul 21 20:49:11.616031 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:11.615970    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:11.714047 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:11.714013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:14.893610 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:14.893582    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6f45cb59b7-pmg28"
Jul 21 20:49:14.925459 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:14.925429    2567 generic.go:356] "Generic (PLEG): container finished" podID="55009324-0e72-44b6-a2df-5652f7aaaaab" containerID="cce3d9625175a3d4d5f0c85a51c34d97679c0b582e658068730737fb6ff3d046" exitCode=0
Jul 21 20:49:14.925617 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:14.925509    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" event={"ID":"55009324-0e72-44b6-a2df-5652f7aaaaab","Type":"ContainerDied","Data":"cce3d9625175a3d4d5f0c85a51c34d97679c0b582e658068730737fb6ff3d046"}
Jul 21 20:49:14.926058 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:14.926039    2567 scope.go:117] "RemoveContainer" containerID="cce3d9625175a3d4d5f0c85a51c34d97679c0b582e658068730737fb6ff3d046"
Jul 21 20:49:14.927182 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:14.927145    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 20:49:14.927290 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:14.927239    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 20:49:14.927341 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:14.927289    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 20:49:15.932763 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:15.932720    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" event={"ID":"55009324-0e72-44b6-a2df-5652f7aaaaab","Type":"ContainerStarted","Data":"ddb3ce53c0fc585cc403c01e077363ef425957b6134fae00a3c8fad2643d4116"}
Jul 21 20:49:16.033777 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:16.033743    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:49:16.035191 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:16.034837    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-bnqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 21 20:49:16.035191 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:16.034895    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" podUID="55009324-0e72-44b6-a2df-5652f7aaaaab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 21 20:49:16.607825 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:16.607771    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:16.698744 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:16.698702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:17.033452 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:17.033420    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-bnqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 21 20:49:17.033853 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:17.033470    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" podUID="55009324-0e72-44b6-a2df-5652f7aaaaab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 21 20:49:18.033630 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:18.033587    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-bnqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 21 20:49:18.034088 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:18.033654    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" podUID="55009324-0e72-44b6-a2df-5652f7aaaaab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 21 20:49:18.414358 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:18.414329    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:49:19.034117 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:19.034073    2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-bnqdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 21 20:49:19.034629 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:19.034143    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr" podUID="55009324-0e72-44b6-a2df-5652f7aaaaab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 21 20:49:20.037637 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:20.037602    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:49:20.038582 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:20.038562    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-bnqdr"
Jul 21 20:49:20.895525 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:20.895487    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-b48b6"
Jul 21 20:49:24.422043 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.422007    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"]
Jul 21 20:49:24.425715 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.425696    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.428540 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.428519    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 20:49:24.429571 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.429555    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k4xb8\""
Jul 21 20:49:24.429661 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.429590    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 20:49:24.429661 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.429607    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 21 20:49:24.440441 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.440407    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"]
Jul 21 20:49:24.458049 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.458009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.458207 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.458056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.458207 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.458082    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.458207 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.458183    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.458348 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.458222    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.458348 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.458263    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mw57\" (UniqueName: \"kubernetes.io/projected/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-kube-api-access-2mw57\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.474006 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.473943    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:24.559053 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.559220 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559061    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.559220 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559098    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mw57\" (UniqueName: \"kubernetes.io/projected/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-kube-api-access-2mw57\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.559220 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.559220 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559153    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.559220 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559177    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.559564 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559540    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.559634 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.559634 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.559598    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.561356 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.561326    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.561596 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.561576    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.572569 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.572539    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mw57\" (UniqueName: \"kubernetes.io/projected/b9d08045-fb6e-43bf-ac85-62f9b109f2f9-kube-api-access-2mw57\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth\" (UID: \"b9d08045-fb6e-43bf-ac85-62f9b109f2f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.735494 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.735394    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:24.879940 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.879911    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"]
Jul 21 20:49:24.882542 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:49:24.882513    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d08045_fb6e_43bf_ac85_62f9b109f2f9.slice/crio-425eb77da35aae6fbf029985d2c9c6a755e7d1c36ed38c7af13236dddd04d342 WatchSource:0}: Error finding container 425eb77da35aae6fbf029985d2c9c6a755e7d1c36ed38c7af13236dddd04d342: Status 404 returned error can't find the container with id 425eb77da35aae6fbf029985d2c9c6a755e7d1c36ed38c7af13236dddd04d342
Jul 21 20:49:24.986602 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:24.986511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth" event={"ID":"b9d08045-fb6e-43bf-ac85-62f9b109f2f9","Type":"ContainerStarted","Data":"425eb77da35aae6fbf029985d2c9c6a755e7d1c36ed38c7af13236dddd04d342"}
Jul 21 20:49:26.999831 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:26.999794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth" event={"ID":"b9d08045-fb6e-43bf-ac85-62f9b109f2f9","Type":"ContainerStarted","Data":"01a4d233b0d557d0eaa76a2fa47d7aea755fb3365a39772e1073e2dc444198a3"}
Jul 21 20:49:27.019231 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:27.019184    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth" podStartSLOduration=1.8248133389999999 podStartE2EDuration="3.019169342s" totalImagesPullingTime="1.194356003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:49:27.017081548 +0000 UTC m=+594.227109972" watchObservedRunningTime="2026-07-21 20:49:27.019169342 +0000 UTC m=+594.229197764"
Jul 21 20:49:33.259224 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:33.259195    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:49:33.260268 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:33.260250    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:49:34.736506 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:34.736472    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:34.736881 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:34.736545    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:34.749089 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:34.749062    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:35.053355 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:35.053268    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth"
Jul 21 20:49:40.042473 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:40.042426    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:54.610120 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.610080    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"]
Jul 21 20:49:54.615081 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.615060    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.619219 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.619200    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 21 20:49:54.626354 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.626333    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"]
Jul 21 20:49:54.649742 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.649712    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:49:54.718409 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.718374    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.718564 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.718417    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.718564 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.718443    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.718564 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.718518    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.718564 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.718556    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96skc\" (UniqueName: \"kubernetes.io/projected/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-kube-api-access-96skc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.718701 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.718599    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819199 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819163    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819395 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819395 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819268    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819395 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819301    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819395 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819358    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819395 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819377    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96skc\" (UniqueName: \"kubernetes.io/projected/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-kube-api-access-96skc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819866 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819689    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819866 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819728    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.819866 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.819772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.821612 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.821580    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.821896 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.821876    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.828446 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.828422    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96skc\" (UniqueName: \"kubernetes.io/projected/7bad3452-89ad-4d26-a105-22b7fdf4f4d8-kube-api-access-96skc\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pjd7n\" (UID: \"7bad3452-89ad-4d26-a105-22b7fdf4f4d8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:54.925711 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:54.925684    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:49:55.060485 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:55.060459    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"]
Jul 21 20:49:55.062145 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:49:55.062113    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bad3452_89ad_4d26_a105_22b7fdf4f4d8.slice/crio-ef2900b913c36e3c3aeac05a0b8037f753d5f4661f58dd777b9bb118ab953b2b WatchSource:0}: Error finding container ef2900b913c36e3c3aeac05a0b8037f753d5f4661f58dd777b9bb118ab953b2b: Status 404 returned error can't find the container with id ef2900b913c36e3c3aeac05a0b8037f753d5f4661f58dd777b9bb118ab953b2b
Jul 21 20:49:55.063431 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:55.063413    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 20:49:55.147267 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:55.147233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n" event={"ID":"7bad3452-89ad-4d26-a105-22b7fdf4f4d8","Type":"ContainerStarted","Data":"ef2900b913c36e3c3aeac05a0b8037f753d5f4661f58dd777b9bb118ab953b2b"}
Jul 21 20:49:56.154329 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:56.154297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n" event={"ID":"7bad3452-89ad-4d26-a105-22b7fdf4f4d8","Type":"ContainerStarted","Data":"621c6f7a0e32851f541cceaacbd68954058af8dec652cba235ae29c8add35771"}
Jul 21 20:49:56.173466 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:49:56.173423    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n" podStartSLOduration=1.960854635 podStartE2EDuration="2.173407684s" totalImagesPullingTime="212.553049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:49:56.172183657 +0000 UTC m=+623.382212117" watchObservedRunningTime="2026-07-21 20:49:56.173407684 +0000 UTC m=+623.383436107"
Jul 21 20:50:04.926736 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:04.926694    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:50:04.927269 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:04.926787    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:50:04.939248 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:04.939218    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:50:05.209768 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:05.209688    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pjd7n"
Jul 21 20:50:06.789547 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:06.789504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:50:06.853295 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:06.853260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:50:10.411261 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.411226    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"]
Jul 21 20:50:10.416149 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.416124    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.419636 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.419609    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 21 20:50:10.435461 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.435429    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"]
Jul 21 20:50:10.457935 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.457882    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.458147 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.457964    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.458147 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.458037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvljq\" (UniqueName: \"kubernetes.io/projected/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-kube-api-access-jvljq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.458275 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.458158    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.458275 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.458240    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.458377 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.458292    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.465180 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.465148    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:50:10.558971 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.558934    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.558971 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.558994    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvljq\" (UniqueName: \"kubernetes.io/projected/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-kube-api-access-jvljq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.559241 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.559113    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.559241 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.559149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.559241 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.559182    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.559413 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.559248    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.559623 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.559597    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.559721 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.559627    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.559721 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.559659    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.561688 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.561664    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.561802 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.561772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.568309 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.568282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvljq\" (UniqueName: \"kubernetes.io/projected/b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7-kube-api-access-jvljq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx\" (UID: \"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.727935 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.727887    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:10.868586 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:10.868532    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"]
Jul 21 20:50:10.871029 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:50:10.870960    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43d70eb_ece3_4b8a_b524_3fc7e0ad2af7.slice/crio-ea687804f15457f13bf466c77dc623125eb1df34dc37b6d758c6b5d46c6210b0 WatchSource:0}: Error finding container ea687804f15457f13bf466c77dc623125eb1df34dc37b6d758c6b5d46c6210b0: Status 404 returned error can't find the container with id ea687804f15457f13bf466c77dc623125eb1df34dc37b6d758c6b5d46c6210b0
Jul 21 20:50:11.230514 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:11.230479    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx" event={"ID":"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7","Type":"ContainerStarted","Data":"a7d1af2447f39881d5f325e72d04d1193683f507f595d833fcc3a631847a83df"}
Jul 21 20:50:11.230703 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:11.230555    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx" event={"ID":"b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7","Type":"ContainerStarted","Data":"ea687804f15457f13bf466c77dc623125eb1df34dc37b6d758c6b5d46c6210b0"}
Jul 21 20:50:11.248668 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:11.248609    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx" podStartSLOduration=1.04367787 podStartE2EDuration="1.248592394s" totalImagesPullingTime="204.914524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:50:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:50:11.247333993 +0000 UTC m=+638.457362411" watchObservedRunningTime="2026-07-21 20:50:11.248592394 +0000 UTC m=+638.458620840"
Jul 21 20:50:20.728272 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:20.728233    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:20.728769 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:20.728297    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:20.741110 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:20.741079    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:21.294760 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:21.294731    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx"
Jul 21 20:50:22.030727 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:22.030679    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:50:24.902881 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:24.902832    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:50:44.773239 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:50:44.773207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:52:12.350343 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:52:12.350256    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:52:12.488345 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:52:12.488306    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:52:13.794744 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:52:13.794715    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:52:14.184896 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:52:14.184859    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:52:28.712499 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:52:28.712460    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:53:04.560032 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.559936    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-4qtwl"]
Jul 21 20:53:04.560573 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.560201    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl" podUID="883e7ff5-8efb-49f8-8fe4-eabca37dbb10" containerName="manager" containerID="cri-o://b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3" gracePeriod=30
Jul 21 20:53:04.674767 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.674728    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl" podUID="883e7ff5-8efb-49f8-8fe4-eabca37dbb10" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.39:8081/readyz\": dial tcp 10.132.0.39:8081: connect: connection refused"
Jul 21 20:53:04.812015 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.811950    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:53:04.837546 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.837522    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert\") pod \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") "
Jul 21 20:53:04.837834 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.837765    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqhqx\" (UniqueName: \"kubernetes.io/projected/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-kube-api-access-jqhqx\") pod \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\" (UID: \"883e7ff5-8efb-49f8-8fe4-eabca37dbb10\") "
Jul 21 20:53:04.840374 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.840348    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "883e7ff5-8efb-49f8-8fe4-eabca37dbb10" (UID: "883e7ff5-8efb-49f8-8fe4-eabca37dbb10"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 20:53:04.840466 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.840347    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-kube-api-access-jqhqx" (OuterVolumeSpecName: "kube-api-access-jqhqx") pod "883e7ff5-8efb-49f8-8fe4-eabca37dbb10" (UID: "883e7ff5-8efb-49f8-8fe4-eabca37dbb10"). InnerVolumeSpecName "kube-api-access-jqhqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:53:04.938899 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.938871    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-webhook-cert\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:53:04.938899 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:04.938899    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqhqx\" (UniqueName: \"kubernetes.io/projected/883e7ff5-8efb-49f8-8fe4-eabca37dbb10-kube-api-access-jqhqx\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:53:05.112670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.112579    2567 generic.go:356] "Generic (PLEG): container finished" podID="883e7ff5-8efb-49f8-8fe4-eabca37dbb10" containerID="b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3" exitCode=0
Jul 21 20:53:05.112670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.112620    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl" event={"ID":"883e7ff5-8efb-49f8-8fe4-eabca37dbb10","Type":"ContainerDied","Data":"b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3"}
Jul 21 20:53:05.112670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.112629    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl"
Jul 21 20:53:05.112670 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.112666    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-4qtwl" event={"ID":"883e7ff5-8efb-49f8-8fe4-eabca37dbb10","Type":"ContainerDied","Data":"2d5f0cf3da431de33de9a0f6f60d79267824dcf0f54e5a6b6072c319236b9965"}
Jul 21 20:53:05.112982 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.112690    2567 scope.go:117] "RemoveContainer" containerID="b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3"
Jul 21 20:53:05.121560 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.121538    2567 scope.go:117] "RemoveContainer" containerID="b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3"
Jul 21 20:53:05.121799 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:53:05.121779    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3\": container with ID starting with b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3 not found: ID does not exist" containerID="b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3"
Jul 21 20:53:05.121874 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.121806    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3"} err="failed to get container status \"b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3\": rpc error: code = NotFound desc = could not find container \"b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3\": container with ID starting with b67036dd41b6ef835e10c34ac81dd80a8add4dc2e03dabedd0ff635dbc0d15e3 not found: ID does not exist"
Jul 21 20:53:05.135351 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.135329    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-4qtwl"]
Jul 21 20:53:05.137389 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.137369    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-4qtwl"]
Jul 21 20:53:05.310783 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:05.310747    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="883e7ff5-8efb-49f8-8fe4-eabca37dbb10" path="/var/lib/kubelet/pods/883e7ff5-8efb-49f8-8fe4-eabca37dbb10/volumes"
Jul 21 20:53:09.580029 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:09.579971    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:53:57.545828 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:57.545795    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:53:57.646880 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:53:57.646842    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:54:06.065854 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.065819    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-z5gxn"]
Jul 21 20:54:06.066318 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.066216    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="883e7ff5-8efb-49f8-8fe4-eabca37dbb10" containerName="manager"
Jul 21 20:54:06.066318 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.066229    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="883e7ff5-8efb-49f8-8fe4-eabca37dbb10" containerName="manager"
Jul 21 20:54:06.066318 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.066301    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="883e7ff5-8efb-49f8-8fe4-eabca37dbb10" containerName="manager"
Jul 21 20:54:06.069293 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.069276    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:06.075696 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.075676    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 20:54:06.075823 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.075676    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 20:54:06.075823 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.075676    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8xkcb\""
Jul 21 20:54:06.086903 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.086874    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-z5gxn"]
Jul 21 20:54:06.133906 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.133872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89d2m\" (UniqueName: \"kubernetes.io/projected/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-kube-api-access-89d2m\") pod \"maas-controller-84f9cc6f45-z5gxn\" (UID: \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\") " pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:06.134075 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.133911    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-webhook-cert\") pod \"maas-controller-84f9cc6f45-z5gxn\" (UID: \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\") " pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:06.235460 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.235418    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89d2m\" (UniqueName: \"kubernetes.io/projected/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-kube-api-access-89d2m\") pod \"maas-controller-84f9cc6f45-z5gxn\" (UID: \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\") " pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:06.235460 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.235466    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-webhook-cert\") pod \"maas-controller-84f9cc6f45-z5gxn\" (UID: \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\") " pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:06.238056 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.238023    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-webhook-cert\") pod \"maas-controller-84f9cc6f45-z5gxn\" (UID: \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\") " pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:06.245563 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.245534    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89d2m\" (UniqueName: \"kubernetes.io/projected/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-kube-api-access-89d2m\") pod \"maas-controller-84f9cc6f45-z5gxn\" (UID: \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\") " pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:06.378695 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.378614    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:06.517540 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:06.517490    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-z5gxn"]
Jul 21 20:54:06.519681 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:54:06.519653    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f1f6c3e_6ca7_4911_aae8_2812c6badaff.slice/crio-1b8e687e15b2688b1773d74106340f2a7c8f58fa362fe141389d6c4006c994f0 WatchSource:0}: Error finding container 1b8e687e15b2688b1773d74106340f2a7c8f58fa362fe141389d6c4006c994f0: Status 404 returned error can't find the container with id 1b8e687e15b2688b1773d74106340f2a7c8f58fa362fe141389d6c4006c994f0
Jul 21 20:54:07.444674 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:07.444637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn" event={"ID":"3f1f6c3e-6ca7-4911-aae8-2812c6badaff","Type":"ContainerStarted","Data":"07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5"}
Jul 21 20:54:07.445166 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:07.444701    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn" event={"ID":"3f1f6c3e-6ca7-4911-aae8-2812c6badaff","Type":"ContainerStarted","Data":"1b8e687e15b2688b1773d74106340f2a7c8f58fa362fe141389d6c4006c994f0"}
Jul 21 20:54:07.445166 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:07.444729    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:07.470115 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:07.470057    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn" podStartSLOduration=0.895189727 podStartE2EDuration="1.470040125s" totalImagesPullingTime="574.850398ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:54:07.467706614 +0000 UTC m=+874.677735064" watchObservedRunningTime="2026-07-21 20:54:07.470040125 +0000 UTC m=+874.680068546"
Jul 21 20:54:18.452374 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:18.452342    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:23.025699 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.025660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:54:23.134184 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.134143    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-z5gxn"]
Jul 21 20:54:23.134405 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.134376    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn" podUID="3f1f6c3e-6ca7-4911-aae8-2812c6badaff" containerName="manager" containerID="cri-o://07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5" gracePeriod=30
Jul 21 20:54:23.379632 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.379607    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:23.484541 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.484502    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-webhook-cert\") pod \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\" (UID: \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\") "
Jul 21 20:54:23.484701 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.484578    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89d2m\" (UniqueName: \"kubernetes.io/projected/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-kube-api-access-89d2m\") pod \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\" (UID: \"3f1f6c3e-6ca7-4911-aae8-2812c6badaff\") "
Jul 21 20:54:23.486767 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.486730    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3f1f6c3e-6ca7-4911-aae8-2812c6badaff" (UID: "3f1f6c3e-6ca7-4911-aae8-2812c6badaff"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 20:54:23.486900 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.486803    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-kube-api-access-89d2m" (OuterVolumeSpecName: "kube-api-access-89d2m") pod "3f1f6c3e-6ca7-4911-aae8-2812c6badaff" (UID: "3f1f6c3e-6ca7-4911-aae8-2812c6badaff"). InnerVolumeSpecName "kube-api-access-89d2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:54:23.529505 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.529468    2567 generic.go:356] "Generic (PLEG): container finished" podID="3f1f6c3e-6ca7-4911-aae8-2812c6badaff" containerID="07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5" exitCode=0
Jul 21 20:54:23.529653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.529520    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn"
Jul 21 20:54:23.529653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.529560    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn" event={"ID":"3f1f6c3e-6ca7-4911-aae8-2812c6badaff","Type":"ContainerDied","Data":"07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5"}
Jul 21 20:54:23.529653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.529622    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-z5gxn" event={"ID":"3f1f6c3e-6ca7-4911-aae8-2812c6badaff","Type":"ContainerDied","Data":"1b8e687e15b2688b1773d74106340f2a7c8f58fa362fe141389d6c4006c994f0"}
Jul 21 20:54:23.529653 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.529647    2567 scope.go:117] "RemoveContainer" containerID="07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5"
Jul 21 20:54:23.538914 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.538893    2567 scope.go:117] "RemoveContainer" containerID="07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5"
Jul 21 20:54:23.539196 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:54:23.539175    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5\": container with ID starting with 07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5 not found: ID does not exist" containerID="07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5"
Jul 21 20:54:23.539253 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.539205    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5"} err="failed to get container status \"07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5\": rpc error: code = NotFound desc = could not find container \"07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5\": container with ID starting with 07e7922b64bccb17f08fe83657a5f3dd68d43312629f09285df429384ba624f5 not found: ID does not exist"
Jul 21 20:54:23.552530 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.552471    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-z5gxn"]
Jul 21 20:54:23.556488 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.556463    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-z5gxn"]
Jul 21 20:54:23.586249 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.586226    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-webhook-cert\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:54:23.586249 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:23.586249    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89d2m\" (UniqueName: \"kubernetes.io/projected/3f1f6c3e-6ca7-4911-aae8-2812c6badaff-kube-api-access-89d2m\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:54:24.251432 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.251387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:54:24.376756 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.376719    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-bfwsb"]
Jul 21 20:54:24.377077 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.377064    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f1f6c3e-6ca7-4911-aae8-2812c6badaff" containerName="manager"
Jul 21 20:54:24.377135 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.377078    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f1f6c3e-6ca7-4911-aae8-2812c6badaff" containerName="manager"
Jul 21 20:54:24.377175 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.377146    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f1f6c3e-6ca7-4911-aae8-2812c6badaff" containerName="manager"
Jul 21 20:54:24.381354 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.381335    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:24.384657 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.384622    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 20:54:24.384657 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.384640    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 20:54:24.384657 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.384647    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8xkcb\""
Jul 21 20:54:24.393015 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.390596    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-bfwsb"]
Jul 21 20:54:24.495459 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.495369    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxvq6\" (UniqueName: \"kubernetes.io/projected/1d31704d-555a-4831-bd20-bc2afd77f6ef-kube-api-access-fxvq6\") pod \"maas-controller-84f9cc6f45-bfwsb\" (UID: \"1d31704d-555a-4831-bd20-bc2afd77f6ef\") " pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:24.495459 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.495405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d31704d-555a-4831-bd20-bc2afd77f6ef-webhook-cert\") pod \"maas-controller-84f9cc6f45-bfwsb\" (UID: \"1d31704d-555a-4831-bd20-bc2afd77f6ef\") " pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:24.596529 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.596492    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxvq6\" (UniqueName: \"kubernetes.io/projected/1d31704d-555a-4831-bd20-bc2afd77f6ef-kube-api-access-fxvq6\") pod \"maas-controller-84f9cc6f45-bfwsb\" (UID: \"1d31704d-555a-4831-bd20-bc2afd77f6ef\") " pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:24.596529 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.596542    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d31704d-555a-4831-bd20-bc2afd77f6ef-webhook-cert\") pod \"maas-controller-84f9cc6f45-bfwsb\" (UID: \"1d31704d-555a-4831-bd20-bc2afd77f6ef\") " pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:24.598980 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.598953    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d31704d-555a-4831-bd20-bc2afd77f6ef-webhook-cert\") pod \"maas-controller-84f9cc6f45-bfwsb\" (UID: \"1d31704d-555a-4831-bd20-bc2afd77f6ef\") " pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:24.605498 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.605473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxvq6\" (UniqueName: \"kubernetes.io/projected/1d31704d-555a-4831-bd20-bc2afd77f6ef-kube-api-access-fxvq6\") pod \"maas-controller-84f9cc6f45-bfwsb\" (UID: \"1d31704d-555a-4831-bd20-bc2afd77f6ef\") " pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:24.696666 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.696625    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:24.833400 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:24.833372    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-84f9cc6f45-bfwsb"]
Jul 21 20:54:24.835686 ip-10-0-130-177 kubenswrapper[2567]: W0721 20:54:24.835661    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d31704d_555a_4831_bd20_bc2afd77f6ef.slice/crio-d31842e1c8fa4675d657dbb7f4bce778e4ca4e15cdf6aba949e0083b6a5ce555 WatchSource:0}: Error finding container d31842e1c8fa4675d657dbb7f4bce778e4ca4e15cdf6aba949e0083b6a5ce555: Status 404 returned error can't find the container with id d31842e1c8fa4675d657dbb7f4bce778e4ca4e15cdf6aba949e0083b6a5ce555
Jul 21 20:54:25.311311 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:25.311278    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f1f6c3e-6ca7-4911-aae8-2812c6badaff" path="/var/lib/kubelet/pods/3f1f6c3e-6ca7-4911-aae8-2812c6badaff/volumes"
Jul 21 20:54:25.543042 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:25.542897    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-bfwsb" event={"ID":"1d31704d-555a-4831-bd20-bc2afd77f6ef","Type":"ContainerStarted","Data":"073762c0320a0e7f54d54935731a7e62d47c64baafee2f764c912f95aad39c53"}
Jul 21 20:54:25.543042 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:25.542951    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-84f9cc6f45-bfwsb" event={"ID":"1d31704d-555a-4831-bd20-bc2afd77f6ef","Type":"ContainerStarted","Data":"d31842e1c8fa4675d657dbb7f4bce778e4ca4e15cdf6aba949e0083b6a5ce555"}
Jul 21 20:54:25.543042 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:25.543028    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:54:25.562700 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:25.562647    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-84f9cc6f45-bfwsb" podStartSLOduration=1.194253598 podStartE2EDuration="1.562631444s" totalImagesPullingTime="368.377846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 20:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 20:54:25.559580772 +0000 UTC m=+892.769609212" watchObservedRunningTime="2026-07-21 20:54:25.562631444 +0000 UTC m=+892.772659866"
Jul 21 20:54:33.296429 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:33.296396    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:54:33.296928 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:33.296900    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:54:36.550109 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:54:36.550078    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-84f9cc6f45-bfwsb"
Jul 21 20:55:07.942004 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:07.941949    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:55:08.040295 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:08.040260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:55:08.151514 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:08.151472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:55:13.990254 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:13.990211    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:55:16.353617 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:16.353581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:55:20.203545 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:20.203504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:55:20.303515 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:20.303479    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:55:46.043839 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:46.043803    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:55:46.144221 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:55:46.144178    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:57:38.808837 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:57:38.808803    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:59:08.917966 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:08.917919    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 20:59:19.074377 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:19.074338    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"]
Jul 21 20:59:19.076814 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:19.074549    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp" podUID="03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a" containerName="manager" containerID="cri-o://a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b" gracePeriod=10
Jul 21 20:59:22.925278 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:22.925252    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:59:22.977802 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:22.977772    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9g6rn\" (UniqueName: \"kubernetes.io/projected/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-kube-api-access-9g6rn\") pod \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\" (UID: \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\") "
Jul 21 20:59:22.977978 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:22.977813    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-extensions-socket-volume\") pod \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\" (UID: \"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a\") "
Jul 21 20:59:22.978246 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:22.978217    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a" (UID: "03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 20:59:22.979919 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:22.979888    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-kube-api-access-9g6rn" (OuterVolumeSpecName: "kube-api-access-9g6rn") pod "03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a" (UID: "03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a"). InnerVolumeSpecName "kube-api-access-9g6rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 20:59:23.078844 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.078754    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-extensions-socket-volume\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:59:23.078844 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.078787    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g6rn\" (UniqueName: \"kubernetes.io/projected/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a-kube-api-access-9g6rn\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 20:59:23.096414 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.096378    2567 generic.go:356] "Generic (PLEG): container finished" podID="03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a" containerID="a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b" exitCode=0
Jul 21 20:59:23.096553 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.096426    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"
Jul 21 20:59:23.096553 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.096440    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp" event={"ID":"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a","Type":"ContainerDied","Data":"a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b"}
Jul 21 20:59:23.096553 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.096494    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp" event={"ID":"03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a","Type":"ContainerDied","Data":"adc825f7fec28be8c8dc49febf58e387cc727433552b64c5b04d230504598571"}
Jul 21 20:59:23.096553 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.096525    2567 scope.go:117] "RemoveContainer" containerID="a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b"
Jul 21 20:59:23.106598 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.106581    2567 scope.go:117] "RemoveContainer" containerID="a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b"
Jul 21 20:59:23.106849 ip-10-0-130-177 kubenswrapper[2567]: E0721 20:59:23.106831    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b\": container with ID starting with a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b not found: ID does not exist" containerID="a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b"
Jul 21 20:59:23.106909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.106856    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b"} err="failed to get container status \"a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b\": rpc error: code = NotFound desc = could not find container \"a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b\": container with ID starting with a704449bddbf9564987fa55e082641acfa28a031497f9bb65019eef6f7544c1b not found: ID does not exist"
Jul 21 20:59:23.120588 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.120563    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"]
Jul 21 20:59:23.126570 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.126545    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gbdpp"]
Jul 21 20:59:23.316909 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:23.316875    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a" path="/var/lib/kubelet/pods/03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a/volumes"
Jul 21 20:59:33.329584 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:33.329553    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 20:59:33.331064 ip-10-0-130-177 kubenswrapper[2567]: I0721 20:59:33.331040    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:00:00.176738 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.176705    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn"]
Jul 21 21:00:00.177132 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.177044    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a" containerName="manager"
Jul 21 21:00:00.177132 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.177056    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a" containerName="manager"
Jul 21 21:00:00.177204 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.177140    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="03e1dfd9-2504-4379-a4e5-21aaa9ce2e7a" containerName="manager"
Jul 21 21:00:00.181263 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.181243    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn"
Jul 21 21:00:00.184107 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.184088    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmths\""
Jul 21 21:00:00.190425 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.190408    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn"
Jul 21 21:00:00.236253 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.236221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn"]
Jul 21 21:00:00.329940 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.329918    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn"]
Jul 21 21:00:00.332020 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:00:00.331966    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2baddef3_e6a1_4567_aaa5_72664ef16034.slice/crio-82b0df1014f646c108aa48d3b60e062566abebc0870a430df5679fae4fb8d91f WatchSource:0}: Error finding container 82b0df1014f646c108aa48d3b60e062566abebc0870a430df5679fae4fb8d91f: Status 404 returned error can't find the container with id 82b0df1014f646c108aa48d3b60e062566abebc0870a430df5679fae4fb8d91f
Jul 21 21:00:00.333849 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:00.333828    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 21:00:01.297696 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:01.297660    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn" event={"ID":"2baddef3-e6a1-4567-aaa5-72664ef16034","Type":"ContainerStarted","Data":"82b0df1014f646c108aa48d3b60e062566abebc0870a430df5679fae4fb8d91f"}
Jul 21 21:00:06.326835 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:06.326725    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn" event={"ID":"2baddef3-e6a1-4567-aaa5-72664ef16034","Type":"ContainerStarted","Data":"e9b0ef33bc554927ed22eca5fa9379f891777817f2427452a574bcf92b574cff"}
Jul 21 21:00:06.361182 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:06.361133    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn" podStartSLOduration=0.972010852 podStartE2EDuration="6.361118644s" totalImagesPullingTime="5.389107792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:06.360449346 +0000 UTC m=+1233.570477779" watchObservedRunningTime="2026-07-21 21:00:06.361118644 +0000 UTC m=+1233.571147064"
Jul 21 21:00:07.333294 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:07.333261    2567 generic.go:356] "Generic (PLEG): container finished" podID="2baddef3-e6a1-4567-aaa5-72664ef16034" containerID="e9b0ef33bc554927ed22eca5fa9379f891777817f2427452a574bcf92b574cff" exitCode=0
Jul 21 21:00:07.333727 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:07.333351    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn" event={"ID":"2baddef3-e6a1-4567-aaa5-72664ef16034","Type":"ContainerDied","Data":"e9b0ef33bc554927ed22eca5fa9379f891777817f2427452a574bcf92b574cff"}
Jul 21 21:00:08.470598 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:08.470572    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn"
Jul 21 21:00:09.343933 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:09.343904    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn"
Jul 21 21:00:09.344094 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:09.343933    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744460-xg7hn" event={"ID":"2baddef3-e6a1-4567-aaa5-72664ef16034","Type":"ContainerDied","Data":"82b0df1014f646c108aa48d3b60e062566abebc0870a430df5679fae4fb8d91f"}
Jul 21 21:00:09.344094 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:09.343974    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82b0df1014f646c108aa48d3b60e062566abebc0870a430df5679fae4fb8d91f"
Jul 21 21:00:20.370806 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.370765    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"]
Jul 21 21:00:20.371216 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.371115    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2baddef3-e6a1-4567-aaa5-72664ef16034" containerName="cleanup"
Jul 21 21:00:20.371216 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.371127    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2baddef3-e6a1-4567-aaa5-72664ef16034" containerName="cleanup"
Jul 21 21:00:20.371216 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.371204    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2baddef3-e6a1-4567-aaa5-72664ef16034" containerName="cleanup"
Jul 21 21:00:20.374223 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.374206    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:20.377310 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.377289    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5rr4s\""
Jul 21 21:00:20.389693 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.389659    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"]
Jul 21 21:00:20.467366 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.467323    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lrfg\" (UniqueName: \"kubernetes.io/projected/2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65-kube-api-access-2lrfg\") pod \"kuadrant-operator-controller-manager-86878c769f-hnp24\" (UID: \"2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:20.467542 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.467468    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hnp24\" (UID: \"2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:20.568786 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.568751    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hnp24\" (UID: \"2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:20.568944 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.568794    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lrfg\" (UniqueName: \"kubernetes.io/projected/2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65-kube-api-access-2lrfg\") pod \"kuadrant-operator-controller-manager-86878c769f-hnp24\" (UID: \"2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:20.569152 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.569127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-hnp24\" (UID: \"2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:20.579015 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.578964    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lrfg\" (UniqueName: \"kubernetes.io/projected/2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65-kube-api-access-2lrfg\") pod \"kuadrant-operator-controller-manager-86878c769f-hnp24\" (UID: \"2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:20.683778 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.683747    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:20.832105 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:20.832075    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"]
Jul 21 21:00:20.834737 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:00:20.834699    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee2bc77_b69b_4fd7_b8b0_e2e3e1a9be65.slice/crio-e63950e673e343e432f505186bada747cffa27180a13bfde5f3305c0bfb279ff WatchSource:0}: Error finding container e63950e673e343e432f505186bada747cffa27180a13bfde5f3305c0bfb279ff: Status 404 returned error can't find the container with id e63950e673e343e432f505186bada747cffa27180a13bfde5f3305c0bfb279ff
Jul 21 21:00:21.408248 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:21.408211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24" event={"ID":"2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65","Type":"ContainerStarted","Data":"c8bd852d4543f6d69531b615a60c7efcf2eee2bfbd91fbe78da25c9c98c8ec35"}
Jul 21 21:00:21.408665 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:21.408263    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24" event={"ID":"2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65","Type":"ContainerStarted","Data":"e63950e673e343e432f505186bada747cffa27180a13bfde5f3305c0bfb279ff"}
Jul 21 21:00:21.408665 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:21.408333    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:00:21.437516 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:21.437462    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24" podStartSLOduration=1.437447386 podStartE2EDuration="1.437447386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:00:21.435779992 +0000 UTC m=+1248.645808634" watchObservedRunningTime="2026-07-21 21:00:21.437447386 +0000 UTC m=+1248.647475807"
Jul 21 21:00:32.415736 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:00:32.415705    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-hnp24"
Jul 21 21:02:58.500421 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:02:58.500383    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:03:00.453912 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:03:00.453871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:03:00.915459 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:03:00.915427    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:03:29.191745 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:03:29.191705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:03:55.448972 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:03:55.448932    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:03:56.502796 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:03:56.502761    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:04:33.363339 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:04:33.363297    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:04:33.367365 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:04:33.367340    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:06:49.284438 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:06:49.284399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:08.666707 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:08.666671    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:30.147277 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:30.147232    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:33.278417 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:33.278383    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:33.366082 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:33.366044    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:33.778290 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:33.778242    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:39.493138 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:39.493103    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:39.583668 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:39.583626    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:39.749979 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:39.749892    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:40.098305 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:40.098221    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:40.669923 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:40.669887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:42.833289 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.833240    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:42.923234 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.923197    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"]
Jul 21 21:07:42.925367 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.925339    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:42.927095 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.927072    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:42.929414 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.929392    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-82e9b33b-serving-cert\""
Jul 21 21:07:42.929543 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.929521    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:07:42.936685 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.936657    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"]
Jul 21 21:07:42.976268 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.976228    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be819963-e2f7-49f2-99e4-ffb2a0b732e0-openshift-service-ca\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:42.976453 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.976382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:42.976453 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:42.976421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm7th\" (UniqueName: \"kubernetes.io/projected/be819963-e2f7-49f2-99e4-ffb2a0b732e0-kube-api-access-pm7th\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.012421 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.012383    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:43.077414 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.077378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.077591 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.077422    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm7th\" (UniqueName: \"kubernetes.io/projected/be819963-e2f7-49f2-99e4-ffb2a0b732e0-kube-api-access-pm7th\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.077591 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.077465    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be819963-e2f7-49f2-99e4-ffb2a0b732e0-openshift-service-ca\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.077591 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:07:43.077512    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-serving-cert: secret "maas-api-e2e-ait-82e9b33b-serving-cert" not found
Jul 21 21:07:43.077718 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:07:43.077596    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls podName:be819963-e2f7-49f2-99e4-ffb2a0b732e0 nodeName:}" failed. No retries permitted until 2026-07-21 21:07:43.577577257 +0000 UTC m=+1690.787605659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls") pod "maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts" (UID: "be819963-e2f7-49f2-99e4-ffb2a0b732e0") : secret "maas-api-e2e-ait-82e9b33b-serving-cert" not found
Jul 21 21:07:43.078273 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.078255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be819963-e2f7-49f2-99e4-ffb2a0b732e0-openshift-service-ca\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.089131 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.089070    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm7th\" (UniqueName: \"kubernetes.io/projected/be819963-e2f7-49f2-99e4-ffb2a0b732e0-kube-api-access-pm7th\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.304078 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.304013    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:43.509449 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.509409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:43.581396 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.581359    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.583885 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.583860    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls\") pod \"maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.609365 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.609328    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:07:43.840932 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.840833    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:43.984417 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.984387    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"]
Jul 21 21:07:43.986255 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:07:43.986229    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe819963_e2f7_49f2_99e4_ffb2a0b732e0.slice/crio-9cfc40dad997aabff5e844a5bcf5806e598384f80a189f6b18f05b00bb7d2744 WatchSource:0}: Error finding container 9cfc40dad997aabff5e844a5bcf5806e598384f80a189f6b18f05b00bb7d2744: Status 404 returned error can't find the container with id 9cfc40dad997aabff5e844a5bcf5806e598384f80a189f6b18f05b00bb7d2744
Jul 21 21:07:43.987544 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:43.987527    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 21:07:44.748903 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:44.748806    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts" event={"ID":"be819963-e2f7-49f2-99e4-ffb2a0b732e0","Type":"ContainerStarted","Data":"bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44"}
Jul 21 21:07:44.748903 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:44.748871    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:07:44.748903 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:44.748895    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts" event={"ID":"be819963-e2f7-49f2-99e4-ffb2a0b732e0","Type":"ContainerStarted","Data":"9cfc40dad997aabff5e844a5bcf5806e598384f80a189f6b18f05b00bb7d2744"}
Jul 21 21:07:44.771249 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:44.771191    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts" podStartSLOduration=2.309617395 podStartE2EDuration="2.771171382s" totalImagesPullingTime="461.553987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:07:44.767374633 +0000 UTC m=+1691.977403055" watchObservedRunningTime="2026-07-21 21:07:44.771171382 +0000 UTC m=+1691.981199805"
Jul 21 21:07:50.759070 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:07:50.759040    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:08:18.981623 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:18.981298    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"]
Jul 21 21:08:18.982202 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:18.981641    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts" podUID="be819963-e2f7-49f2-99e4-ffb2a0b732e0" containerName="maas-api" containerID="cri-o://bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44" gracePeriod=30
Jul 21 21:08:19.011818 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.011790    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:19.231227 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.231205    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:08:19.401777 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.401695    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm7th\" (UniqueName: \"kubernetes.io/projected/be819963-e2f7-49f2-99e4-ffb2a0b732e0-kube-api-access-pm7th\") pod \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") "
Jul 21 21:08:19.401777 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.401744    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls\") pod \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") "
Jul 21 21:08:19.401977 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.401787    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be819963-e2f7-49f2-99e4-ffb2a0b732e0-openshift-service-ca\") pod \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\" (UID: \"be819963-e2f7-49f2-99e4-ffb2a0b732e0\") "
Jul 21 21:08:19.402194 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.402164    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be819963-e2f7-49f2-99e4-ffb2a0b732e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "be819963-e2f7-49f2-99e4-ffb2a0b732e0" (UID: "be819963-e2f7-49f2-99e4-ffb2a0b732e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:08:19.404010 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.403965    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "be819963-e2f7-49f2-99e4-ffb2a0b732e0" (UID: "be819963-e2f7-49f2-99e4-ffb2a0b732e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:08:19.404010 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.403981    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be819963-e2f7-49f2-99e4-ffb2a0b732e0-kube-api-access-pm7th" (OuterVolumeSpecName: "kube-api-access-pm7th") pod "be819963-e2f7-49f2-99e4-ffb2a0b732e0" (UID: "be819963-e2f7-49f2-99e4-ffb2a0b732e0"). InnerVolumeSpecName "kube-api-access-pm7th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:08:19.503314 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.503278    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm7th\" (UniqueName: \"kubernetes.io/projected/be819963-e2f7-49f2-99e4-ffb2a0b732e0-kube-api-access-pm7th\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:08:19.503314 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.503307    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be819963-e2f7-49f2-99e4-ffb2a0b732e0-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:08:19.503314 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.503317    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be819963-e2f7-49f2-99e4-ffb2a0b732e0-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:08:19.939608 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.939573    2567 generic.go:356] "Generic (PLEG): container finished" podID="be819963-e2f7-49f2-99e4-ffb2a0b732e0" containerID="bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44" exitCode=0
Jul 21 21:08:19.939792 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.939620    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"
Jul 21 21:08:19.939792 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.939654    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts" event={"ID":"be819963-e2f7-49f2-99e4-ffb2a0b732e0","Type":"ContainerDied","Data":"bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44"}
Jul 21 21:08:19.939792 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.939703    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts" event={"ID":"be819963-e2f7-49f2-99e4-ffb2a0b732e0","Type":"ContainerDied","Data":"9cfc40dad997aabff5e844a5bcf5806e598384f80a189f6b18f05b00bb7d2744"}
Jul 21 21:08:19.939792 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.939724    2567 scope.go:117] "RemoveContainer" containerID="bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44"
Jul 21 21:08:19.949417 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.949397    2567 scope.go:117] "RemoveContainer" containerID="bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44"
Jul 21 21:08:19.949669 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:08:19.949640    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44\": container with ID starting with bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44 not found: ID does not exist" containerID="bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44"
Jul 21 21:08:19.949726 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.949667    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44"} err="failed to get container status \"bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44\": rpc error: code = NotFound desc = could not find container \"bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44\": container with ID starting with bda1a8293fef545f7b5e6de222d37d4078a0e787c2f19aae3ca5e7f72a54bc44 not found: ID does not exist"
Jul 21 21:08:19.962740 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.962715    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"]
Jul 21 21:08:19.965590 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:19.965567    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-82e9b33b-8f7bbc958-8ntts"]
Jul 21 21:08:21.311285 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:21.311252    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be819963-e2f7-49f2-99e4-ffb2a0b732e0" path="/var/lib/kubelet/pods/be819963-e2f7-49f2-99e4-ffb2a0b732e0/volumes"
Jul 21 21:08:23.615468 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:23.615399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:23.782162 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:23.782130    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:24.025424 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:24.025387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:24.343647 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:24.343551    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:24.437529 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:24.437491    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:24.537671 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:24.537632    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:24.904896 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:24.904863    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:25.201572 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:25.201535    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:27.179823 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.179782    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:27.401107 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.401073    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"]
Jul 21 21:08:27.401446 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.401433    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be819963-e2f7-49f2-99e4-ffb2a0b732e0" containerName="maas-api"
Jul 21 21:08:27.401495 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.401447    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="be819963-e2f7-49f2-99e4-ffb2a0b732e0" containerName="maas-api"
Jul 21 21:08:27.401536 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.401500    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="be819963-e2f7-49f2-99e4-ffb2a0b732e0" containerName="maas-api"
Jul 21 21:08:27.404887 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.404867    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.407179 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.407152    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-0f699c57-serving-cert\""
Jul 21 21:08:27.407438 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.407417    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:08:27.415809 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.415783    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"]
Jul 21 21:08:27.438277 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.438187    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:27.572138 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.572099    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w94rt\" (UniqueName: \"kubernetes.io/projected/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-kube-api-access-w94rt\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.572328 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.572176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-maas-api-tls\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.572328 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.572210    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-openshift-service-ca\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.602630 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.602595    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:27.672878 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.672835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-maas-api-tls\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.673103 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.672890    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-openshift-service-ca\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.673103 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.672932    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w94rt\" (UniqueName: \"kubernetes.io/projected/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-kube-api-access-w94rt\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.673644 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.673618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-openshift-service-ca\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.675266 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.675247    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-maas-api-tls\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.685732 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.685708    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w94rt\" (UniqueName: \"kubernetes.io/projected/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-kube-api-access-w94rt\") pod \"maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:27.716941 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:27.716851    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:28.025900 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:28.025798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:28.063365 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:28.063329    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"]
Jul 21 21:08:28.064537 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:08:28.064511    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbac71eaf_b4a0_4e6c_bc2e_77289bc27810.slice/crio-b98ffaa14d44cd97d7c3c3e63fcfbdef112a241deb4f76f2c017a959a16f60bb WatchSource:0}: Error finding container b98ffaa14d44cd97d7c3c3e63fcfbdef112a241deb4f76f2c017a959a16f60bb: Status 404 returned error can't find the container with id b98ffaa14d44cd97d7c3c3e63fcfbdef112a241deb4f76f2c017a959a16f60bb
Jul 21 21:08:28.124643 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:28.124609    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:08:28.991526 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:28.991491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd" event={"ID":"bac71eaf-b4a0-4e6c-bc2e-77289bc27810","Type":"ContainerStarted","Data":"f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a"}
Jul 21 21:08:28.991913 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:28.991547    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd" event={"ID":"bac71eaf-b4a0-4e6c-bc2e-77289bc27810","Type":"ContainerStarted","Data":"b98ffaa14d44cd97d7c3c3e63fcfbdef112a241deb4f76f2c017a959a16f60bb"}
Jul 21 21:08:28.991913 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:28.991585    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:29.009748 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:29.009690    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd" podStartSLOduration=1.7286063170000001 podStartE2EDuration="2.009674967s" totalImagesPullingTime="281.06865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:08:29.009068698 +0000 UTC m=+1736.219097183" watchObservedRunningTime="2026-07-21 21:08:29.009674967 +0000 UTC m=+1736.219703388"
Jul 21 21:08:35.002551 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:35.002518    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:08:50.902594 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:08:50.902557    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:02.900487 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:02.900448    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"]
Jul 21 21:09:02.900955 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:02.900724    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd" podUID="bac71eaf-b4a0-4e6c-bc2e-77289bc27810" containerName="maas-api" containerID="cri-o://f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a" gracePeriod=30
Jul 21 21:09:02.945382 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:02.945356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:03.160596 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.160539    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:09:03.163492 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.163467    2567 generic.go:356] "Generic (PLEG): container finished" podID="bac71eaf-b4a0-4e6c-bc2e-77289bc27810" containerID="f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a" exitCode=0
Jul 21 21:09:03.163605 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.163502    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"
Jul 21 21:09:03.163605 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.163553    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd" event={"ID":"bac71eaf-b4a0-4e6c-bc2e-77289bc27810","Type":"ContainerDied","Data":"f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a"}
Jul 21 21:09:03.163680 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.163610    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd" event={"ID":"bac71eaf-b4a0-4e6c-bc2e-77289bc27810","Type":"ContainerDied","Data":"b98ffaa14d44cd97d7c3c3e63fcfbdef112a241deb4f76f2c017a959a16f60bb"}
Jul 21 21:09:03.163680 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.163633    2567 scope.go:117] "RemoveContainer" containerID="f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a"
Jul 21 21:09:03.171886 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.171868    2567 scope.go:117] "RemoveContainer" containerID="f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a"
Jul 21 21:09:03.172156 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:09:03.172137    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a\": container with ID starting with f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a not found: ID does not exist" containerID="f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a"
Jul 21 21:09:03.172212 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.172163    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a"} err="failed to get container status \"f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a\": rpc error: code = NotFound desc = could not find container \"f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a\": container with ID starting with f938f065eb76d14ce93d0ce8f94dd2dadad2e650ba8d723373c831ad4137707a not found: ID does not exist"
Jul 21 21:09:03.300410 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.300374    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-maas-api-tls\") pod \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") "
Jul 21 21:09:03.300592 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.300420    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w94rt\" (UniqueName: \"kubernetes.io/projected/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-kube-api-access-w94rt\") pod \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") "
Jul 21 21:09:03.300592 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.300438    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-openshift-service-ca\") pod \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\" (UID: \"bac71eaf-b4a0-4e6c-bc2e-77289bc27810\") "
Jul 21 21:09:03.300825 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.300803    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bac71eaf-b4a0-4e6c-bc2e-77289bc27810" (UID: "bac71eaf-b4a0-4e6c-bc2e-77289bc27810"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:09:03.302551 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.302531    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-kube-api-access-w94rt" (OuterVolumeSpecName: "kube-api-access-w94rt") pod "bac71eaf-b4a0-4e6c-bc2e-77289bc27810" (UID: "bac71eaf-b4a0-4e6c-bc2e-77289bc27810"). InnerVolumeSpecName "kube-api-access-w94rt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:09:03.302621 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.302598    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bac71eaf-b4a0-4e6c-bc2e-77289bc27810" (UID: "bac71eaf-b4a0-4e6c-bc2e-77289bc27810"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:09:03.401731 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.401693    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w94rt\" (UniqueName: \"kubernetes.io/projected/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-kube-api-access-w94rt\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:03.401731 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.401733    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:03.401955 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.401842    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bac71eaf-b4a0-4e6c-bc2e-77289bc27810-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:03.481343 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.481315    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"]
Jul 21 21:09:03.486232 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:03.486207    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-0f699c57-665f4f4468-fdtvd"]
Jul 21 21:09:05.311583 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:05.311548    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac71eaf-b4a0-4e6c-bc2e-77289bc27810" path="/var/lib/kubelet/pods/bac71eaf-b4a0-4e6c-bc2e-77289bc27810/volumes"
Jul 21 21:09:07.320265 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:07.320226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:07.484061 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:07.484020    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:07.714252 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:07.714218    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:07.962021 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:07.959906    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:08.271111 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.271078    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"]
Jul 21 21:09:08.271480 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.271459    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac71eaf-b4a0-4e6c-bc2e-77289bc27810" containerName="maas-api"
Jul 21 21:09:08.271480 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.271478    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac71eaf-b4a0-4e6c-bc2e-77289bc27810" containerName="maas-api"
Jul 21 21:09:08.271726 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.271562    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac71eaf-b4a0-4e6c-bc2e-77289bc27810" containerName="maas-api"
Jul 21 21:09:08.274453 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.274436    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.276881 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.276855    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-21b7cb95-openshift-default-dockercfg-z7qst\""
Jul 21 21:09:08.288280 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.288256    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"]
Jul 21 21:09:08.300922 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.300896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:08.446668 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-podinfo\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.446668 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446676    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-token\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.447127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446705    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istiod-ca-cert\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.447127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446826    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-credential-socket\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.447127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446865    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-certs\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.447127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446902    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-socket\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.447127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-data\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.447127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446947    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-envoy\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.447127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.446969    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbbh2\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-kube-api-access-dbbh2\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.503999 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.503946    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:08.548095 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-credential-socket\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548095 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548058    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-certs\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548095 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548082    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-socket\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548358 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548105    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-data\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548358 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548122    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-envoy\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548358 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548137    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbbh2\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-kube-api-access-dbbh2\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548358 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548178    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-podinfo\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548358 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-token\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548358 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548324    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istiod-ca-cert\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548662 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548506    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-certs\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548662 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548514    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-credential-socket\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548662 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548590    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-data\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.548903 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.548663    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-socket\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.549110 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.549089    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istiod-ca-cert\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.550539 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.550523    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-envoy\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.550768 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.550749    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-podinfo\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.556707 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.556670    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbbh2\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-kube-api-access-dbbh2\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.556832 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.556816    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-token\") pod \"e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") " pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.584848 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.584812    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:08.725482 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.725399    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"]
Jul 21 21:09:08.727809 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:09:08.727776    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd322404e_e1f9_479c_bf84_fb91cc8bdf89.slice/crio-aa464489adea93f52cc8569039f7d22a21a3b5624bb3b34ae255fd844f4778be WatchSource:0}: Error finding container aa464489adea93f52cc8569039f7d22a21a3b5624bb3b34ae255fd844f4778be: Status 404 returned error can't find the container with id aa464489adea93f52cc8569039f7d22a21a3b5624bb3b34ae255fd844f4778be
Jul 21 21:09:08.730019 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.729971    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:09:08.730127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.730066    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:09:08.730127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:08.730106    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:09:09.196871 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:09.196825    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" event={"ID":"d322404e-e1f9-479c-bf84-fb91cc8bdf89","Type":"ContainerStarted","Data":"bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e"}
Jul 21 21:09:09.197058 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:09.196891    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" event={"ID":"d322404e-e1f9-479c-bf84-fb91cc8bdf89","Type":"ContainerStarted","Data":"aa464489adea93f52cc8569039f7d22a21a3b5624bb3b34ae255fd844f4778be"}
Jul 21 21:09:09.218817 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:09.218766    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" podStartSLOduration=1.218751747 podStartE2EDuration="1.218751747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:09:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:09:09.217269964 +0000 UTC m=+1776.427298423" watchObservedRunningTime="2026-07-21 21:09:09.218751747 +0000 UTC m=+1776.428780227"
Jul 21 21:09:09.585801 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:09.585713    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:09.590554 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:09.590529    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:10.201782 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.201752    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:10.202641 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.202622    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:10.613125 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.613049    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:10.699637 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.699606    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"]
Jul 21 21:09:10.703203 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.703180    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:10.704778 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.704753    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:10.706294 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.706269    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-21b7cb95-serving-cert\""
Jul 21 21:09:10.706610 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.706595    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:09:10.713513 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.713469    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"]
Jul 21 21:09:10.766925 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.766892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42cd5af9-2d4c-4e42-8302-b0605a673228-openshift-service-ca\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:10.767106 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.766935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:10.767106 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.766965    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96xjf\" (UniqueName: \"kubernetes.io/projected/42cd5af9-2d4c-4e42-8302-b0605a673228-kube-api-access-96xjf\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:10.793776 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.793737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:10.867634 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.867554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42cd5af9-2d4c-4e42-8302-b0605a673228-openshift-service-ca\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:10.867634 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.867592    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:10.867634 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.867611    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96xjf\" (UniqueName: \"kubernetes.io/projected/42cd5af9-2d4c-4e42-8302-b0605a673228-kube-api-access-96xjf\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:10.867859 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:09:10.867716    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-serving-cert: secret "maas-api-e2e-ait-21b7cb95-serving-cert" not found
Jul 21 21:09:10.867859 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:09:10.867780    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls podName:42cd5af9-2d4c-4e42-8302-b0605a673228 nodeName:}" failed. No retries permitted until 2026-07-21 21:09:11.367764319 +0000 UTC m=+1778.577792741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls") pod "maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx" (UID: "42cd5af9-2d4c-4e42-8302-b0605a673228") : secret "maas-api-e2e-ait-21b7cb95-serving-cert" not found
Jul 21 21:09:10.868206 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.868187    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42cd5af9-2d4c-4e42-8302-b0605a673228-openshift-service-ca\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:10.883225 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:10.883201    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96xjf\" (UniqueName: \"kubernetes.io/projected/42cd5af9-2d4c-4e42-8302-b0605a673228-kube-api-access-96xjf\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:11.176545 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:11.176514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:11.309885 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:11.309833    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:11.371519 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:11.371483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:11.373821 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:11.373794    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls\") pod \"maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:11.415467 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:11.415433    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:11.615617 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:11.615527    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:11.781065 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:11.781041    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"]
Jul 21 21:09:11.783467 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:09:11.783438    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42cd5af9_2d4c_4e42_8302_b0605a673228.slice/crio-ad7da0057e70b9fe3e9d895619d534510740a24edc5dc42f44564a53be07f732 WatchSource:0}: Error finding container ad7da0057e70b9fe3e9d895619d534510740a24edc5dc42f44564a53be07f732: Status 404 returned error can't find the container with id ad7da0057e70b9fe3e9d895619d534510740a24edc5dc42f44564a53be07f732
Jul 21 21:09:12.214558 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:12.214518    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx" event={"ID":"42cd5af9-2d4c-4e42-8302-b0605a673228","Type":"ContainerStarted","Data":"ad7da0057e70b9fe3e9d895619d534510740a24edc5dc42f44564a53be07f732"}
Jul 21 21:09:13.221035 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:13.220972    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx" event={"ID":"42cd5af9-2d4c-4e42-8302-b0605a673228","Type":"ContainerStarted","Data":"4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7"}
Jul 21 21:09:13.221411 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:13.221184    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:13.242794 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:13.242691    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx" podStartSLOduration=2.700328565 podStartE2EDuration="3.242673781s" totalImagesPullingTime="542.345216ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:09:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:09:13.241758933 +0000 UTC m=+1780.451787375" watchObservedRunningTime="2026-07-21 21:09:13.242673781 +0000 UTC m=+1780.452702205"
Jul 21 21:09:19.231889 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:19.231858    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:25.203124 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:25.203088    2567 patch_prober.go:28] interesting pod/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 21 21:09:25.203586 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:25.203156    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 21 21:09:25.292285 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:25.292203    2567 generic.go:356] "Generic (PLEG): container finished" podID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerID="bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e" exitCode=0
Jul 21 21:09:25.292448 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:25.292275    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" event={"ID":"d322404e-e1f9-479c-bf84-fb91cc8bdf89","Type":"ContainerDied","Data":"bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e"}
Jul 21 21:09:25.292707 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:25.292693    2567 scope.go:117] "RemoveContainer" containerID="bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e"
Jul 21 21:09:25.293730 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:25.293702    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:09:25.293836 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:25.293793    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:09:25.293905 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:25.293848    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:09:26.299781 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:26.299743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" event={"ID":"d322404e-e1f9-479c-bf84-fb91cc8bdf89","Type":"ContainerStarted","Data":"394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8"}
Jul 21 21:09:26.585935 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:26.585836    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:26.586127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:26.586103    2567 patch_prober.go:28] interesting pod/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 21 21:09:26.586181 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:26.586151    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 21 21:09:27.585886 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:27.585849    2567 patch_prober.go:28] interesting pod/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 21 21:09:27.586277 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:27.585910    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 21 21:09:28.585300 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:28.585255    2567 patch_prober.go:28] interesting pod/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 21 21:09:28.585452 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:28.585319    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 21 21:09:29.586239 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:29.586200    2567 patch_prober.go:28] interesting pod/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 21 21:09:29.586636 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:29.586270    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 21 21:09:30.589621 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:30.589592    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:30.590053 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:30.589878    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:30.590604 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:30.590586    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:33.398286 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:33.398257    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:09:33.402812 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:33.402788    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:09:45.910107 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:45.910062    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:46.008963 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:46.008922    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:47.274974 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.274930    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:47.375796 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.375757    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:47.517680 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.517643    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"]
Jul 21 21:09:47.517922 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.517896    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx" podUID="42cd5af9-2d4c-4e42-8302-b0605a673228" containerName="maas-api" containerID="cri-o://4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7" gracePeriod=30
Jul 21 21:09:47.540207 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.540176    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:47.776883 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.776859    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:47.906440 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.906355    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls\") pod \"42cd5af9-2d4c-4e42-8302-b0605a673228\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") "
Jul 21 21:09:47.906440 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.906391    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96xjf\" (UniqueName: \"kubernetes.io/projected/42cd5af9-2d4c-4e42-8302-b0605a673228-kube-api-access-96xjf\") pod \"42cd5af9-2d4c-4e42-8302-b0605a673228\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") "
Jul 21 21:09:47.906440 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.906429    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42cd5af9-2d4c-4e42-8302-b0605a673228-openshift-service-ca\") pod \"42cd5af9-2d4c-4e42-8302-b0605a673228\" (UID: \"42cd5af9-2d4c-4e42-8302-b0605a673228\") "
Jul 21 21:09:47.906846 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.906819    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42cd5af9-2d4c-4e42-8302-b0605a673228-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42cd5af9-2d4c-4e42-8302-b0605a673228" (UID: "42cd5af9-2d4c-4e42-8302-b0605a673228"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:09:47.908564 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.908536    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42cd5af9-2d4c-4e42-8302-b0605a673228" (UID: "42cd5af9-2d4c-4e42-8302-b0605a673228"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:09:47.908564 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:47.908535    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42cd5af9-2d4c-4e42-8302-b0605a673228-kube-api-access-96xjf" (OuterVolumeSpecName: "kube-api-access-96xjf") pod "42cd5af9-2d4c-4e42-8302-b0605a673228" (UID: "42cd5af9-2d4c-4e42-8302-b0605a673228"). InnerVolumeSpecName "kube-api-access-96xjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:09:48.007414 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.007371    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42cd5af9-2d4c-4e42-8302-b0605a673228-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:48.007414 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.007406    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96xjf\" (UniqueName: \"kubernetes.io/projected/42cd5af9-2d4c-4e42-8302-b0605a673228-kube-api-access-96xjf\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:48.007414 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.007420    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42cd5af9-2d4c-4e42-8302-b0605a673228-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:48.413011 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.412964    2567 generic.go:356] "Generic (PLEG): container finished" podID="42cd5af9-2d4c-4e42-8302-b0605a673228" containerID="4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7" exitCode=0
Jul 21 21:09:48.413424 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.413023    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"
Jul 21 21:09:48.413424 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.413022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx" event={"ID":"42cd5af9-2d4c-4e42-8302-b0605a673228","Type":"ContainerDied","Data":"4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7"}
Jul 21 21:09:48.413424 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.413136    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx" event={"ID":"42cd5af9-2d4c-4e42-8302-b0605a673228","Type":"ContainerDied","Data":"ad7da0057e70b9fe3e9d895619d534510740a24edc5dc42f44564a53be07f732"}
Jul 21 21:09:48.413424 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.413158    2567 scope.go:117] "RemoveContainer" containerID="4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7"
Jul 21 21:09:48.424447 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.424327    2567 scope.go:117] "RemoveContainer" containerID="4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7"
Jul 21 21:09:48.424974 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:09:48.424949    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7\": container with ID starting with 4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7 not found: ID does not exist" containerID="4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7"
Jul 21 21:09:48.425094 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.424998    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7"} err="failed to get container status \"4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7\": rpc error: code = NotFound desc = could not find container \"4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7\": container with ID starting with 4fc30792cc0fb036c254d5368dac3dd94d74920cbc54a6ebe6af2688c4fb5bf7 not found: ID does not exist"
Jul 21 21:09:48.440395 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.440369    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"]
Jul 21 21:09:48.444632 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:48.444609    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-21b7cb95-775bcc9d4f-d6vmx"]
Jul 21 21:09:49.310973 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:49.310938    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42cd5af9-2d4c-4e42-8302-b0605a673228" path="/var/lib/kubelet/pods/42cd5af9-2d4c-4e42-8302-b0605a673228/volumes"
Jul 21 21:09:51.612457 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:51.612418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:52.877486 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:52.877441    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"]
Jul 21 21:09:52.878013 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:52.877670    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy" containerID="cri-o://394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8" gracePeriod=30
Jul 21 21:09:52.904733 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:52.904691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:53.140862 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:53.140771    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:53.363902 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:53.363870    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:53.699780 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:53.699738    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:54.175480 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:54.175448    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:56.539715 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.539674    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:56.670366 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.670331    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"]
Jul 21 21:09:56.670692 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.670680    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42cd5af9-2d4c-4e42-8302-b0605a673228" containerName="maas-api"
Jul 21 21:09:56.670740 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.670693    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="42cd5af9-2d4c-4e42-8302-b0605a673228" containerName="maas-api"
Jul 21 21:09:56.670775 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.670760    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="42cd5af9-2d4c-4e42-8302-b0605a673228" containerName="maas-api"
Jul 21 21:09:56.675340 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.675317    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.676690 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.676650    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:56.679146 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.679128    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:09:56.681189 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.681169    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-febaebdc-serving-cert\""
Jul 21 21:09:56.684673 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.684652    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-openshift-service-ca\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.684783 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.684693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vktm\" (UniqueName: \"kubernetes.io/projected/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-kube-api-access-9vktm\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.684783 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.684731    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.688071 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.688049    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"]
Jul 21 21:09:56.785723 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.785679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-openshift-service-ca\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.785723 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.785727    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vktm\" (UniqueName: \"kubernetes.io/projected/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-kube-api-access-9vktm\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.785938 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.785759    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.785938 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:09:56.785878    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-serving-cert: secret "maas-api-e2e-derive-febaebdc-serving-cert" not found
Jul 21 21:09:56.786048 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:09:56.785951    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls podName:bf0585f7-1cb7-40db-a2c9-cc18fdf211c4 nodeName:}" failed. No retries permitted until 2026-07-21 21:09:57.285931035 +0000 UTC m=+1824.495959435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls") pod "maas-api-e2e-derive-febaebdc-8564cf5895-s27hc" (UID: "bf0585f7-1cb7-40db-a2c9-cc18fdf211c4") : secret "maas-api-e2e-derive-febaebdc-serving-cert" not found
Jul 21 21:09:56.786330 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.786312    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-openshift-service-ca\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.800048 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.799959    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:56.802094 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.802071    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vktm\" (UniqueName: \"kubernetes.io/projected/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-kube-api-access-9vktm\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:56.896205 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:56.896165    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:57.289753 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:57.289712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:57.292198 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:57.292173    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls\") pod \"maas-api-e2e-derive-febaebdc-8564cf5895-s27hc\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:57.323568 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:57.323531    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:57.424783 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:57.424746    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:09:57.585697 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:57.585612    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:57.725240 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:57.725211    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"]
Jul 21 21:09:57.726865 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:09:57.726838    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0585f7_1cb7_40db_a2c9_cc18fdf211c4.slice/crio-f344b4df11c680ab3cac97432e61ac248c245a8e5a8d709b3cbcbf75077e4b26 WatchSource:0}: Error finding container f344b4df11c680ab3cac97432e61ac248c245a8e5a8d709b3cbcbf75077e4b26: Status 404 returned error can't find the container with id f344b4df11c680ab3cac97432e61ac248c245a8e5a8d709b3cbcbf75077e4b26
Jul 21 21:09:58.130487 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.130459    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:58.198480 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198442    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-podinfo\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.198628 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198495    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-envoy\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.198628 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198532    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istiod-ca-cert\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.198628 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198595    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-credential-socket\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.198628 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198623    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-data\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.198864 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198669    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbbh2\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-kube-api-access-dbbh2\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.198864 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198688    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-socket\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.198864 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198706    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-certs\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.198864 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198743    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-token\") pod \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\" (UID: \"d322404e-e1f9-479c-bf84-fb91cc8bdf89\") "
Jul 21 21:09:58.199088 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198928    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:09:58.199088 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.198944    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:09:58.199088 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.199078    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istiod-ca-cert\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.199258 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.199097    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-credential-socket\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.199258 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.199247    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:09:58.199374 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.199262    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:09:58.199584 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.199547    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-data" (OuterVolumeSpecName: "istio-data") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:09:58.200893 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.200866    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:09:58.201031 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.200897    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 21:09:58.201031 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.200909    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-kube-api-access-dbbh2" (OuterVolumeSpecName: "kube-api-access-dbbh2") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "kube-api-access-dbbh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:09:58.201031 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.200959    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-token" (OuterVolumeSpecName: "istio-token") pod "d322404e-e1f9-479c-bf84-fb91cc8bdf89" (UID: "d322404e-e1f9-479c-bf84-fb91cc8bdf89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:09:58.300145 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.300108    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-data\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.300145 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.300139    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbbh2\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-kube-api-access-dbbh2\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.300145 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.300149    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-socket\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.300379 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.300162    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-workload-certs\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.300379 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.300171    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-token\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.300379 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.300180    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-podinfo\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.300379 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.300188    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d322404e-e1f9-479c-bf84-fb91cc8bdf89-istio-envoy\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:09:58.468754 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.468664    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc" event={"ID":"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4","Type":"ContainerStarted","Data":"14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c"}
Jul 21 21:09:58.468754 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.468721    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc" event={"ID":"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4","Type":"ContainerStarted","Data":"f344b4df11c680ab3cac97432e61ac248c245a8e5a8d709b3cbcbf75077e4b26"}
Jul 21 21:09:58.468754 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.468753    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:09:58.470410 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.470386    2567 generic.go:356] "Generic (PLEG): container finished" podID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerID="394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8" exitCode=0
Jul 21 21:09:58.470513 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.470432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" event={"ID":"d322404e-e1f9-479c-bf84-fb91cc8bdf89","Type":"ContainerDied","Data":"394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8"}
Jul 21 21:09:58.470513 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.470477    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4" event={"ID":"d322404e-e1f9-479c-bf84-fb91cc8bdf89","Type":"ContainerDied","Data":"aa464489adea93f52cc8569039f7d22a21a3b5624bb3b34ae255fd844f4778be"}
Jul 21 21:09:58.470513 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.470499    2567 scope.go:117] "RemoveContainer" containerID="394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8"
Jul 21 21:09:58.470626 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.470443    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"
Jul 21 21:09:58.483748 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.483730    2567 scope.go:117] "RemoveContainer" containerID="bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e"
Jul 21 21:09:58.486117 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.486071    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc" podStartSLOduration=1.876427184 podStartE2EDuration="2.486055372s" totalImagesPullingTime="609.628188ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:09:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:09:58.484846699 +0000 UTC m=+1825.694875121" watchObservedRunningTime="2026-07-21 21:09:58.486055372 +0000 UTC m=+1825.696083791"
Jul 21 21:09:58.492474 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.492456    2567 scope.go:117] "RemoveContainer" containerID="394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8"
Jul 21 21:09:58.492720 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:09:58.492699    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8\": container with ID starting with 394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8 not found: ID does not exist" containerID="394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8"
Jul 21 21:09:58.492789 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.492728    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8"} err="failed to get container status \"394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8\": rpc error: code = NotFound desc = could not find container \"394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8\": container with ID starting with 394ede050b9292ff9839c20af6b9735bcf0774320b4b00406ca16ac9db000ac8 not found: ID does not exist"
Jul 21 21:09:58.492789 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.492744    2567 scope.go:117] "RemoveContainer" containerID="bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e"
Jul 21 21:09:58.493035 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:09:58.493016    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e\": container with ID starting with bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e not found: ID does not exist" containerID="bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e"
Jul 21 21:09:58.493079 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.493041    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e"} err="failed to get container status \"bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e\": rpc error: code = NotFound desc = could not find container \"bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e\": container with ID starting with bf6fdaaace3cd9e5a1088705730402a08812ed56efd0625a9b45f71fc78ae45e not found: ID does not exist"
Jul 21 21:09:58.503790 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.503768    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"]
Jul 21 21:09:58.507573 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:58.507553    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-21b7cb95-openshift-default-64975949c7-j8jj4"]
Jul 21 21:09:59.311762 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:09:59.311731    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" path="/var/lib/kubelet/pods/d322404e-e1f9-479c-bf84-fb91cc8bdf89/volumes"
Jul 21 21:10:04.482350 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:04.482319    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:10:32.053225 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.053181    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"]
Jul 21 21:10:32.054039 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.053961    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc" podUID="bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" containerName="maas-api" containerID="cri-o://14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c" gracePeriod=30
Jul 21 21:10:32.085445 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.085403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:32.297043 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.297014    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:10:32.406138 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.406054    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vktm\" (UniqueName: \"kubernetes.io/projected/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-kube-api-access-9vktm\") pod \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") "
Jul 21 21:10:32.406138 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.406098    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-openshift-service-ca\") pod \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") "
Jul 21 21:10:32.406368 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.406172    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls\") pod \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\" (UID: \"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4\") "
Jul 21 21:10:32.406504 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.406466    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" (UID: "bf0585f7-1cb7-40db-a2c9-cc18fdf211c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:10:32.408268 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.408248    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" (UID: "bf0585f7-1cb7-40db-a2c9-cc18fdf211c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:10:32.408359 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.408339    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-kube-api-access-9vktm" (OuterVolumeSpecName: "kube-api-access-9vktm") pod "bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" (UID: "bf0585f7-1cb7-40db-a2c9-cc18fdf211c4"). InnerVolumeSpecName "kube-api-access-9vktm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:10:32.507679 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.507647    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vktm\" (UniqueName: \"kubernetes.io/projected/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-kube-api-access-9vktm\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:10:32.507679 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.507672    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:10:32.507679 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.507683    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:10:32.646706 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.646674    2567 generic.go:356] "Generic (PLEG): container finished" podID="bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" containerID="14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c" exitCode=0
Jul 21 21:10:32.646866 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.646715    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"
Jul 21 21:10:32.646866 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.646742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc" event={"ID":"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4","Type":"ContainerDied","Data":"14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c"}
Jul 21 21:10:32.646866 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.646785    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc" event={"ID":"bf0585f7-1cb7-40db-a2c9-cc18fdf211c4","Type":"ContainerDied","Data":"f344b4df11c680ab3cac97432e61ac248c245a8e5a8d709b3cbcbf75077e4b26"}
Jul 21 21:10:32.646866 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.646806    2567 scope.go:117] "RemoveContainer" containerID="14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c"
Jul 21 21:10:32.656371 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.656352    2567 scope.go:117] "RemoveContainer" containerID="14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c"
Jul 21 21:10:32.656623 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:10:32.656603    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c\": container with ID starting with 14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c not found: ID does not exist" containerID="14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c"
Jul 21 21:10:32.656684 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.656632    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c"} err="failed to get container status \"14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c\": rpc error: code = NotFound desc = could not find container \"14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c\": container with ID starting with 14f39aab4fdb4907b9d6696c88dafbce28e34c5904551fde84002e881b95be2c not found: ID does not exist"
Jul 21 21:10:32.672757 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.672734    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"]
Jul 21 21:10:32.677721 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:32.677698    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-febaebdc-8564cf5895-s27hc"]
Jul 21 21:10:33.311029 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:33.310970    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" path="/var/lib/kubelet/pods/bf0585f7-1cb7-40db-a2c9-cc18fdf211c4/volumes"
Jul 21 21:10:36.537375 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:36.537340    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:36.741382 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:36.741346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:36.964730 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:36.964696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:37.474821 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:37.474780    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:37.564666 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:37.564630    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:37.663613 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:37.663572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:39.176546 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:39.176509    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:45.133380 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:45.133344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:45.204878 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:45.204839    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:45.303462 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:45.303423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:45.401841 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:45.401588    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:10:45.858893 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:10:45.858855    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:11.857503 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:11.857421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:18.486446 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:18.486407    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:18.671935 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:18.671897    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:18.865056 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:18.864945    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:34.683250 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:34.683205    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:40.608801 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:40.608760    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:40.682423 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:40.682384    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:40.774693 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:40.774663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:40.877690 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:40.877592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:45.967280 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:45.967245    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:46.035935 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:46.035898    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:51.371355 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:51.371321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:52.209703 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:52.209654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:58.955609 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:58.955573    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:59.065438 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:59.065399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:59.178582 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:59.178552    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:11:59.282443 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:11:59.282350    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:00.090837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:00.090794    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:00.204252 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:00.204190    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:00.292073 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:00.292039    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:02.054945 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:02.054909    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:08.030823 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:08.030784    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:08.097374 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:08.097338    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:08.732614 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:08.732582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:15.368977 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:15.368937    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:15.471453 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:15.471414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:15.616426 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:15.616385    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:16.025223 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:16.025168    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:16.125900 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:16.125864    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:19.120568 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.120528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:19.225307 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225272    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"]
Jul 21 21:12:19.225618 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225606    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" containerName="maas-api"
Jul 21 21:12:19.225660 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225619    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" containerName="maas-api"
Jul 21 21:12:19.225660 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225640    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy"
Jul 21 21:12:19.225660 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225646    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy"
Jul 21 21:12:19.225660 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225653    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy"
Jul 21 21:12:19.225660 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225659    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy"
Jul 21 21:12:19.225810 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225711    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy"
Jul 21 21:12:19.225810 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225718    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d322404e-e1f9-479c-bf84-fb91cc8bdf89" containerName="istio-proxy"
Jul 21 21:12:19.225810 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.225726    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf0585f7-1cb7-40db-a2c9-cc18fdf211c4" containerName="maas-api"
Jul 21 21:12:19.228652 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.228630    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.231532 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.231509    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-99bde6d0-serving-cert\""
Jul 21 21:12:19.231661 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.231534    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:12:19.238188 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.238164    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:19.240752 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.240716    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"]
Jul 21 21:12:19.335844 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.335798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:19.366824 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.366788    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v9wz\" (UniqueName: \"kubernetes.io/projected/82187068-04d5-4d52-a41b-ada6dcc97cdc-kube-api-access-9v9wz\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.367013 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.366880    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82187068-04d5-4d52-a41b-ada6dcc97cdc-maas-api-tls\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.367013 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.366936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82187068-04d5-4d52-a41b-ada6dcc97cdc-openshift-service-ca\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.432371 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.432329    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:19.467715 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.467684    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v9wz\" (UniqueName: \"kubernetes.io/projected/82187068-04d5-4d52-a41b-ada6dcc97cdc-kube-api-access-9v9wz\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.467885 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.467730    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82187068-04d5-4d52-a41b-ada6dcc97cdc-maas-api-tls\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.467885 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.467756    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82187068-04d5-4d52-a41b-ada6dcc97cdc-openshift-service-ca\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.468360 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.468337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82187068-04d5-4d52-a41b-ada6dcc97cdc-openshift-service-ca\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.470152 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.470129    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82187068-04d5-4d52-a41b-ada6dcc97cdc-maas-api-tls\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.476098 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.476074    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v9wz\" (UniqueName: \"kubernetes.io/projected/82187068-04d5-4d52-a41b-ada6dcc97cdc-kube-api-access-9v9wz\") pod \"maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.540884 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.540851    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:19.686691 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.686658    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"]
Jul 21 21:12:19.687717 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:12:19.687674    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82187068_04d5_4d52_a41b_ada6dcc97cdc.slice/crio-a7a2d6fef81357a65867e08cd70e513de519d9ec08e4c2c88e1cc2cc6e8710ef WatchSource:0}: Error finding container a7a2d6fef81357a65867e08cd70e513de519d9ec08e4c2c88e1cc2cc6e8710ef: Status 404 returned error can't find the container with id a7a2d6fef81357a65867e08cd70e513de519d9ec08e4c2c88e1cc2cc6e8710ef
Jul 21 21:12:19.756498 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.756416    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:19.856457 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:19.856424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:20.209695 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:20.209663    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv" event={"ID":"82187068-04d5-4d52-a41b-ada6dcc97cdc","Type":"ContainerStarted","Data":"a7a2d6fef81357a65867e08cd70e513de519d9ec08e4c2c88e1cc2cc6e8710ef"}
Jul 21 21:12:21.216418 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:21.216381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv" event={"ID":"82187068-04d5-4d52-a41b-ada6dcc97cdc","Type":"ContainerStarted","Data":"546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60"}
Jul 21 21:12:21.216797 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:21.216442    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:21.239471 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:21.239410    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv" podStartSLOduration=1.761703757 podStartE2EDuration="2.239389882s" totalImagesPullingTime="477.686125ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:12:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:12:21.235599156 +0000 UTC m=+1968.445627608" watchObservedRunningTime="2026-07-21 21:12:21.239389882 +0000 UTC m=+1968.449418304"
Jul 21 21:12:21.519489 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:21.519395    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:21.621297 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:21.621248    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:23.493450 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:23.493409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:24.203065 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:24.202953    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:24.295058 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:24.295007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:24.636077 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:24.635977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:27.227379 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:27.227348    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:30.729835 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:30.729794    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:30.800997 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:30.800953    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:41.631687 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.631648    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"]
Jul 21 21:12:41.632255 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.631943    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv" podUID="82187068-04d5-4d52-a41b-ada6dcc97cdc" containerName="maas-api" containerID="cri-o://546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60" gracePeriod=30
Jul 21 21:12:41.665832 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.665785    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:41.875496 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.875472    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:41.962190 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.962152    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82187068-04d5-4d52-a41b-ada6dcc97cdc-maas-api-tls\") pod \"82187068-04d5-4d52-a41b-ada6dcc97cdc\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") "
Jul 21 21:12:41.962373 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.962254    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82187068-04d5-4d52-a41b-ada6dcc97cdc-openshift-service-ca\") pod \"82187068-04d5-4d52-a41b-ada6dcc97cdc\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") "
Jul 21 21:12:41.962373 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.962351    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v9wz\" (UniqueName: \"kubernetes.io/projected/82187068-04d5-4d52-a41b-ada6dcc97cdc-kube-api-access-9v9wz\") pod \"82187068-04d5-4d52-a41b-ada6dcc97cdc\" (UID: \"82187068-04d5-4d52-a41b-ada6dcc97cdc\") "
Jul 21 21:12:41.962556 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.962537    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82187068-04d5-4d52-a41b-ada6dcc97cdc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "82187068-04d5-4d52-a41b-ada6dcc97cdc" (UID: "82187068-04d5-4d52-a41b-ada6dcc97cdc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:12:41.962630 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.962595    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/82187068-04d5-4d52-a41b-ada6dcc97cdc-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:12:41.964300 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.964273    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82187068-04d5-4d52-a41b-ada6dcc97cdc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "82187068-04d5-4d52-a41b-ada6dcc97cdc" (UID: "82187068-04d5-4d52-a41b-ada6dcc97cdc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:12:41.964399 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:41.964350    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82187068-04d5-4d52-a41b-ada6dcc97cdc-kube-api-access-9v9wz" (OuterVolumeSpecName: "kube-api-access-9v9wz") pod "82187068-04d5-4d52-a41b-ada6dcc97cdc" (UID: "82187068-04d5-4d52-a41b-ada6dcc97cdc"). InnerVolumeSpecName "kube-api-access-9v9wz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:12:42.063630 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.063596    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/82187068-04d5-4d52-a41b-ada6dcc97cdc-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:12:42.063630 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.063626    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v9wz\" (UniqueName: \"kubernetes.io/projected/82187068-04d5-4d52-a41b-ada6dcc97cdc-kube-api-access-9v9wz\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:12:42.329456 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.329371    2567 generic.go:356] "Generic (PLEG): container finished" podID="82187068-04d5-4d52-a41b-ada6dcc97cdc" containerID="546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60" exitCode=0
Jul 21 21:12:42.329456 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.329418    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"
Jul 21 21:12:42.329632 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.329454    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv" event={"ID":"82187068-04d5-4d52-a41b-ada6dcc97cdc","Type":"ContainerDied","Data":"546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60"}
Jul 21 21:12:42.329632 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.329511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv" event={"ID":"82187068-04d5-4d52-a41b-ada6dcc97cdc","Type":"ContainerDied","Data":"a7a2d6fef81357a65867e08cd70e513de519d9ec08e4c2c88e1cc2cc6e8710ef"}
Jul 21 21:12:42.329632 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.329534    2567 scope.go:117] "RemoveContainer" containerID="546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60"
Jul 21 21:12:42.340247 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.340223    2567 scope.go:117] "RemoveContainer" containerID="546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60"
Jul 21 21:12:42.340512 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:12:42.340493    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60\": container with ID starting with 546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60 not found: ID does not exist" containerID="546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60"
Jul 21 21:12:42.340608 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.340520    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60"} err="failed to get container status \"546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60\": rpc error: code = NotFound desc = could not find container \"546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60\": container with ID starting with 546464e41e36a5388c28da8a6137312b5b7c065f548cc01a4c3bfbeb5e6ebf60 not found: ID does not exist"
Jul 21 21:12:42.352956 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.352927    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"]
Jul 21 21:12:42.359575 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:42.359533    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-99bde6d0-6d545868bd-89vqv"]
Jul 21 21:12:43.311645 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:43.311603    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82187068-04d5-4d52-a41b-ada6dcc97cdc" path="/var/lib/kubelet/pods/82187068-04d5-4d52-a41b-ada6dcc97cdc/volumes"
Jul 21 21:12:46.211090 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:46.211052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:46.652601 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:46.652510    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:52.721933 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:52.721896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:52.830998 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:52.830950    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:52.998130 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:52.998032    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:53.287167 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:53.287077    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:53.422784 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:53.422749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:53.518326 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:53.518292    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:53.646612 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:53.646523    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:12:58.038393 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:12:58.038357    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:00.302032 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:00.301970    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:00.513645 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:00.513613    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:00.629591 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:00.629512    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:00.763319 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:00.763274    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:06.618410 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:06.618363    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:06.692350 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:06.692295    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:13.601780 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:13.601733    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:14.107261 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:14.106974    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:14.574732 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:14.574692    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:17.176164 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.176122    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:17.311701 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.311670    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"]
Jul 21 21:13:17.312024 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.312011    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82187068-04d5-4d52-a41b-ada6dcc97cdc" containerName="maas-api"
Jul 21 21:13:17.312082 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.312025    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="82187068-04d5-4d52-a41b-ada6dcc97cdc" containerName="maas-api"
Jul 21 21:13:17.312082 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.312081    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="82187068-04d5-4d52-a41b-ada6dcc97cdc" containerName="maas-api"
Jul 21 21:13:17.316384 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.316350    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.322872 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.322293    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-371611dd-serving-cert\""
Jul 21 21:13:17.322872 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.322531    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:13:17.325095 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.325071    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"]
Jul 21 21:13:17.345405 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.345373    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:17.370809 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.370783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.370917 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.370816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxnsx\" (UniqueName: \"kubernetes.io/projected/21b1552e-1896-433b-8921-53c56a5f6db9-kube-api-access-dxnsx\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.370968 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.370921    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b1552e-1896-433b-8921-53c56a5f6db9-openshift-service-ca\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.471961 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.471846    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b1552e-1896-433b-8921-53c56a5f6db9-openshift-service-ca\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.471961 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.471933    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.471961 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.471958    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxnsx\" (UniqueName: \"kubernetes.io/projected/21b1552e-1896-433b-8921-53c56a5f6db9-kube-api-access-dxnsx\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.472291 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:13:17.472096    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-serving-cert: secret "maas-api-e2e-mt-371611dd-serving-cert" not found
Jul 21 21:13:17.472291 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:13:17.472181    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls podName:21b1552e-1896-433b-8921-53c56a5f6db9 nodeName:}" failed. No retries permitted until 2026-07-21 21:13:17.972159933 +0000 UTC m=+2025.182188335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls") pod "maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d" (UID: "21b1552e-1896-433b-8921-53c56a5f6db9") : secret "maas-api-e2e-mt-371611dd-serving-cert" not found
Jul 21 21:13:17.472532 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.472511    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b1552e-1896-433b-8921-53c56a5f6db9-openshift-service-ca\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.484346 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.484320    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxnsx\" (UniqueName: \"kubernetes.io/projected/21b1552e-1896-433b-8921-53c56a5f6db9-kube-api-access-dxnsx\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.486877 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.486850    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:17.768435 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.768342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:17.976728 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.976694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:17.979251 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:17.979224    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls\") pod \"maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:18.104189 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:18.104114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:18.205686 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:18.205645    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:18.230586 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:18.230554    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:18.575844 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:18.575819    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"]
Jul 21 21:13:18.577825 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:13:18.577797    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b1552e_1896_433b_8921_53c56a5f6db9.slice/crio-60b9713ccf8c95af2dcd64b71ec553afb1032b1a006e13ab5a15f5c54256f799 WatchSource:0}: Error finding container 60b9713ccf8c95af2dcd64b71ec553afb1032b1a006e13ab5a15f5c54256f799: Status 404 returned error can't find the container with id 60b9713ccf8c95af2dcd64b71ec553afb1032b1a006e13ab5a15f5c54256f799
Jul 21 21:13:18.579231 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:18.579211    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 21:13:19.526953 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:19.526916    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d" event={"ID":"21b1552e-1896-433b-8921-53c56a5f6db9","Type":"ContainerStarted","Data":"ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949"}
Jul 21 21:13:19.527439 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:19.526966    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d" event={"ID":"21b1552e-1896-433b-8921-53c56a5f6db9","Type":"ContainerStarted","Data":"60b9713ccf8c95af2dcd64b71ec553afb1032b1a006e13ab5a15f5c54256f799"}
Jul 21 21:13:19.527439 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:19.527018    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:19.546947 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:19.546904    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d" podStartSLOduration=1.967506174 podStartE2EDuration="2.546890469s" totalImagesPullingTime="579.384295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:13:19.546318875 +0000 UTC m=+2026.756347316" watchObservedRunningTime="2026-07-21 21:13:19.546890469 +0000 UTC m=+2026.756918889"
Jul 21 21:13:21.687030 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:21.686977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:13:25.537574 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:25.537540    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:13:41.038635 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:13:41.038599    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:14:26.218839 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.218753    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"]
Jul 21 21:14:26.219332 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.219047    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d" podUID="21b1552e-1896-433b-8921-53c56a5f6db9" containerName="maas-api" containerID="cri-o://ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949" gracePeriod=30
Jul 21 21:14:26.309794 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.309749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:14:26.471884 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.471827    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:14:26.560905 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.560867    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b1552e-1896-433b-8921-53c56a5f6db9-openshift-service-ca\") pod \"21b1552e-1896-433b-8921-53c56a5f6db9\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") "
Jul 21 21:14:26.561171 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.560912    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls\") pod \"21b1552e-1896-433b-8921-53c56a5f6db9\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") "
Jul 21 21:14:26.561171 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.561075    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxnsx\" (UniqueName: \"kubernetes.io/projected/21b1552e-1896-433b-8921-53c56a5f6db9-kube-api-access-dxnsx\") pod \"21b1552e-1896-433b-8921-53c56a5f6db9\" (UID: \"21b1552e-1896-433b-8921-53c56a5f6db9\") "
Jul 21 21:14:26.561345 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.561314    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21b1552e-1896-433b-8921-53c56a5f6db9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "21b1552e-1896-433b-8921-53c56a5f6db9" (UID: "21b1552e-1896-433b-8921-53c56a5f6db9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:14:26.563004 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.562962    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "21b1552e-1896-433b-8921-53c56a5f6db9" (UID: "21b1552e-1896-433b-8921-53c56a5f6db9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:14:26.563091 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.563058    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21b1552e-1896-433b-8921-53c56a5f6db9-kube-api-access-dxnsx" (OuterVolumeSpecName: "kube-api-access-dxnsx") pod "21b1552e-1896-433b-8921-53c56a5f6db9" (UID: "21b1552e-1896-433b-8921-53c56a5f6db9"). InnerVolumeSpecName "kube-api-access-dxnsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:14:26.661919 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.661886    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21b1552e-1896-433b-8921-53c56a5f6db9-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:14:26.661919 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.661914    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxnsx\" (UniqueName: \"kubernetes.io/projected/21b1552e-1896-433b-8921-53c56a5f6db9-kube-api-access-dxnsx\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:14:26.661919 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.661924    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21b1552e-1896-433b-8921-53c56a5f6db9-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:14:26.876572 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.876484    2567 generic.go:356] "Generic (PLEG): container finished" podID="21b1552e-1896-433b-8921-53c56a5f6db9" containerID="ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949" exitCode=0
Jul 21 21:14:26.876572 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.876527    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"
Jul 21 21:14:26.876817 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.876574    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d" event={"ID":"21b1552e-1896-433b-8921-53c56a5f6db9","Type":"ContainerDied","Data":"ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949"}
Jul 21 21:14:26.876817 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.876645    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d" event={"ID":"21b1552e-1896-433b-8921-53c56a5f6db9","Type":"ContainerDied","Data":"60b9713ccf8c95af2dcd64b71ec553afb1032b1a006e13ab5a15f5c54256f799"}
Jul 21 21:14:26.876817 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.876676    2567 scope.go:117] "RemoveContainer" containerID="ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949"
Jul 21 21:14:26.885898 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.885882    2567 scope.go:117] "RemoveContainer" containerID="ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949"
Jul 21 21:14:26.886170 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:14:26.886152    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949\": container with ID starting with ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949 not found: ID does not exist" containerID="ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949"
Jul 21 21:14:26.886226 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.886178    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949"} err="failed to get container status \"ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949\": rpc error: code = NotFound desc = could not find container \"ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949\": container with ID starting with ef17ac7458606147b1139f8a39ddd4589cab88ae34a4a3a132c6c5526bd7f949 not found: ID does not exist"
Jul 21 21:14:26.899864 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.899841    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"]
Jul 21 21:14:26.904297 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:26.904264    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-371611dd-855b9d4fb8-mld9d"]
Jul 21 21:14:27.312498 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:27.312464    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21b1552e-1896-433b-8921-53c56a5f6db9" path="/var/lib/kubelet/pods/21b1552e-1896-433b-8921-53c56a5f6db9/volumes"
Jul 21 21:14:30.822646 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:30.822602    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:14:33.236587 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:33.236550    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:14:33.429917 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:33.429887    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:14:33.435418 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:33.435375    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:14:38.268199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:38.268162    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:14:46.898256 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:46.898207    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:14:46.989695 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:14:46.989659    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:14.890565 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:14.890528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:15.705152 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:15.705118    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:21.589888 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:21.589851    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:21.656920 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:21.656882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:21.890502 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:21.890409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:22.131611 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:22.131573    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:23.282477 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:23.282430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:23.377731 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:23.377696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:23.477309 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:23.477276    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:24.290042 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:24.290007    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:24.387581 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:24.387541    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:24.488007 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:24.487961    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:25.966159 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:25.966130    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:32.010930 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:32.010893    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:32.110684 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:32.110642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:32.712799 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:32.712762    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:38.705534 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:38.705490    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:38.812835 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:38.812795    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:38.906654 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:38.906618    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:49.454145 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:49.454102    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:15:49.808554 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:15:49.808455    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:04.273717 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:04.273675    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:10.856811 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:10.856776    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:17.372317 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:17.372277    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:17.470388 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:17.470354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:17.751496 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:17.751452    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:18.072918 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:18.072836    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:18.224907 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:18.224877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:21.121155 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.121115    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:21.218969 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.218932    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:21.220227 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.220199    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"]
Jul 21 21:16:21.220611 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.220597    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21b1552e-1896-433b-8921-53c56a5f6db9" containerName="maas-api"
Jul 21 21:16:21.220660 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.220614    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="21b1552e-1896-433b-8921-53c56a5f6db9" containerName="maas-api"
Jul 21 21:16:21.220699 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.220686    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="21b1552e-1896-433b-8921-53c56a5f6db9" containerName="maas-api"
Jul 21 21:16:21.223717 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.223700    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.226950 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.226930    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:16:21.230450 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.230386    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-85e235-serving-cert\""
Jul 21 21:16:21.244914 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.244892    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"]
Jul 21 21:16:21.303302 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.303266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:21.335477 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.335443    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plqqt\" (UniqueName: \"kubernetes.io/projected/024788f8-bb9c-498b-a729-65a43652b798-kube-api-access-plqqt\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.335641 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.335510    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024788f8-bb9c-498b-a729-65a43652b798-maas-api-tls\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.335641 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.335591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024788f8-bb9c-498b-a729-65a43652b798-openshift-service-ca\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.437115 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.437079    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plqqt\" (UniqueName: \"kubernetes.io/projected/024788f8-bb9c-498b-a729-65a43652b798-kube-api-access-plqqt\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.437315 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.437145    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024788f8-bb9c-498b-a729-65a43652b798-maas-api-tls\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.437315 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.437234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024788f8-bb9c-498b-a729-65a43652b798-openshift-service-ca\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.437902 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.437874    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024788f8-bb9c-498b-a729-65a43652b798-openshift-service-ca\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.439945 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.439921    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024788f8-bb9c-498b-a729-65a43652b798-maas-api-tls\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.448320 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.448292    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plqqt\" (UniqueName: \"kubernetes.io/projected/024788f8-bb9c-498b-a729-65a43652b798-kube-api-access-plqqt\") pod \"maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.486153 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.486121    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:21.534058 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.534025    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:21.673125 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.673098    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"]
Jul 21 21:16:21.674905 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:16:21.674877    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024788f8_bb9c_498b_a729_65a43652b798.slice/crio-091d040918ed271be966fc72623d7684b0372e298a742a93239afae1740fab6a WatchSource:0}: Error finding container 091d040918ed271be966fc72623d7684b0372e298a742a93239afae1740fab6a: Status 404 returned error can't find the container with id 091d040918ed271be966fc72623d7684b0372e298a742a93239afae1740fab6a
Jul 21 21:16:21.796300 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.796202    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:21.898525 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:21.898493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:22.477942 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:22.477908    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2" event={"ID":"024788f8-bb9c-498b-a729-65a43652b798","Type":"ContainerStarted","Data":"06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2"}
Jul 21 21:16:22.478416 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:22.477970    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2" event={"ID":"024788f8-bb9c-498b-a729-65a43652b798","Type":"ContainerStarted","Data":"091d040918ed271be966fc72623d7684b0372e298a742a93239afae1740fab6a"}
Jul 21 21:16:22.478416 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:22.478014    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:22.496647 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:22.496595    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2" podStartSLOduration=1.091421462 podStartE2EDuration="1.496578511s" totalImagesPullingTime="405.157049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:16:22.495204195 +0000 UTC m=+2209.705232618" watchObservedRunningTime="2026-07-21 21:16:22.496578511 +0000 UTC m=+2209.706606932"
Jul 21 21:16:25.721800 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:25.721753    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:26.062626 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:26.062535    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:26.415100 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:26.415059    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:26.613772 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:26.613732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:28.489215 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:28.489186    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:16:29.713561 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.713504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:29.790918 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.790885    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"]
Jul 21 21:16:29.794345 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.794322    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:29.794628 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.794587    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:29.796772 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.796750    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c15522-serving-cert\""
Jul 21 21:16:29.806811 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.806785    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"]
Jul 21 21:16:29.868089 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.868050    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:29.917095 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.917066    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0382fea5-3f00-4995-a1c9-55f3b8057482-maas-api-tls\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:29.917259 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.917123    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrhjl\" (UniqueName: \"kubernetes.io/projected/0382fea5-3f00-4995-a1c9-55f3b8057482-kube-api-access-zrhjl\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:29.917259 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:29.917194    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0382fea5-3f00-4995-a1c9-55f3b8057482-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:30.005718 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.005633    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:30.017731 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.017705    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0382fea5-3f00-4995-a1c9-55f3b8057482-maas-api-tls\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:30.017864 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.017755    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrhjl\" (UniqueName: \"kubernetes.io/projected/0382fea5-3f00-4995-a1c9-55f3b8057482-kube-api-access-zrhjl\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:30.017864 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.017780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0382fea5-3f00-4995-a1c9-55f3b8057482-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:30.018430 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.018410    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0382fea5-3f00-4995-a1c9-55f3b8057482-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:30.020175 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.020157    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0382fea5-3f00-4995-a1c9-55f3b8057482-maas-api-tls\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:30.030288 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.030268    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrhjl\" (UniqueName: \"kubernetes.io/projected/0382fea5-3f00-4995-a1c9-55f3b8057482-kube-api-access-zrhjl\") pod \"maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:30.105440 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.105414    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:30.244178 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.244152    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"]
Jul 21 21:16:30.246442 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:16:30.246412    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0382fea5_3f00_4995_a1c9_55f3b8057482.slice/crio-cf6ad732de474020da3f4c7fe02a261a949a0637cdc69bb64eefb1a57fc6ccf9 WatchSource:0}: Error finding container cf6ad732de474020da3f4c7fe02a261a949a0637cdc69bb64eefb1a57fc6ccf9: Status 404 returned error can't find the container with id cf6ad732de474020da3f4c7fe02a261a949a0637cdc69bb64eefb1a57fc6ccf9
Jul 21 21:16:30.468803 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.468766    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:30.528758 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.528712    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb" event={"ID":"0382fea5-3f00-4995-a1c9-55f3b8057482","Type":"ContainerStarted","Data":"cf6ad732de474020da3f4c7fe02a261a949a0637cdc69bb64eefb1a57fc6ccf9"}
Jul 21 21:16:30.580465 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:30.580428    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:31.535879 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:31.535831    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb" event={"ID":"0382fea5-3f00-4995-a1c9-55f3b8057482","Type":"ContainerStarted","Data":"81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe"}
Jul 21 21:16:31.536329 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:31.535970    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:31.555186 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:31.555140    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb" podStartSLOduration=2.087890243 podStartE2EDuration="2.555126433s" totalImagesPullingTime="467.23619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:16:31.553765094 +0000 UTC m=+2218.763793517" watchObservedRunningTime="2026-07-21 21:16:31.555126433 +0000 UTC m=+2218.765154854"
Jul 21 21:16:34.168971 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:34.168934    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:37.546576 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:37.546545    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:16:42.587599 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:42.587569    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:42.687261 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:42.687214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:48.679542 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:48.679507    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:51.251326 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:51.251268    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:16:51.353308 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:16:51.353273    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:17:00.831717 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:17:00.831636    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:17:30.927419 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:17:30.927375    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:01.527552 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:01.527518    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:01.589698 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:01.589660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:09.344037 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:09.343997    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:10.272028 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.271980    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:10.401551 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.401517    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"]
Jul 21 21:18:10.404973 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.404951    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"
Jul 21 21:18:10.407703 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.407682    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmths\""
Jul 21 21:18:10.413664 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.413641    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"]
Jul 21 21:18:10.492189 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.492152    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/103d7312-d386-4f63-8047-3560d296ae3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-85e235-t864z\" (UID: \"103d7312-d386-4f63-8047-3560d296ae3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"
Jul 21 21:18:10.593022 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.592905    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/103d7312-d386-4f63-8047-3560d296ae3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-85e235-t864z\" (UID: \"103d7312-d386-4f63-8047-3560d296ae3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"
Jul 21 21:18:10.593515 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.593496    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/103d7312-d386-4f63-8047-3560d296ae3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-85e235-t864z\" (UID: \"103d7312-d386-4f63-8047-3560d296ae3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"
Jul 21 21:18:10.717236 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.717192    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"
Jul 21 21:18:10.851878 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.851850    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"]
Jul 21 21:18:10.853693 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:18:10.853661    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103d7312_d386_4f63_8047_3560d296ae3d.slice/crio-3474f279e2549db900eab465560b05a0ff31722eacd7ce3532f7c6bab7048f9b WatchSource:0}: Error finding container 3474f279e2549db900eab465560b05a0ff31722eacd7ce3532f7c6bab7048f9b: Status 404 returned error can't find the container with id 3474f279e2549db900eab465560b05a0ff31722eacd7ce3532f7c6bab7048f9b
Jul 21 21:18:10.923277 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:10.923238    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:11.025400 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:11.025362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:11.068340 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:11.068310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z" event={"ID":"103d7312-d386-4f63-8047-3560d296ae3d","Type":"ContainerStarted","Data":"3474f279e2549db900eab465560b05a0ff31722eacd7ce3532f7c6bab7048f9b"}
Jul 21 21:18:12.074899 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:12.074861    2567 generic.go:356] "Generic (PLEG): container finished" podID="103d7312-d386-4f63-8047-3560d296ae3d" containerID="01bb80a61261b7f2595be5701c17359e592957e2a71880d5d8ec0502defdcbe8" exitCode=0
Jul 21 21:18:12.075286 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:12.074954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z" event={"ID":"103d7312-d386-4f63-8047-3560d296ae3d","Type":"ContainerDied","Data":"01bb80a61261b7f2595be5701c17359e592957e2a71880d5d8ec0502defdcbe8"}
Jul 21 21:18:13.210930 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:13.210909    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"
Jul 21 21:18:13.318362 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:13.318335    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/103d7312-d386-4f63-8047-3560d296ae3d-maas-api-service-ca\") pod \"103d7312-d386-4f63-8047-3560d296ae3d\" (UID: \"103d7312-d386-4f63-8047-3560d296ae3d\") "
Jul 21 21:18:13.318669 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:13.318645    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/103d7312-d386-4f63-8047-3560d296ae3d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "103d7312-d386-4f63-8047-3560d296ae3d" (UID: "103d7312-d386-4f63-8047-3560d296ae3d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:18:13.419611 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:13.419564    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/103d7312-d386-4f63-8047-3560d296ae3d-maas-api-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:18:14.085598 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:14.085567    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"
Jul 21 21:18:14.085763 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:14.085562    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z" event={"ID":"103d7312-d386-4f63-8047-3560d296ae3d","Type":"ContainerDied","Data":"3474f279e2549db900eab465560b05a0ff31722eacd7ce3532f7c6bab7048f9b"}
Jul 21 21:18:14.085763 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:14.085694    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3474f279e2549db900eab465560b05a0ff31722eacd7ce3532f7c6bab7048f9b"
Jul 21 21:18:20.829234 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:20.829196    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"]
Jul 21 21:18:20.829809 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:20.829451    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2" podUID="024788f8-bb9c-498b-a729-65a43652b798" containerName="maas-api" containerID="cri-o://06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2" gracePeriod=30
Jul 21 21:18:20.857748 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:20.857703    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:21.083427 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.083368    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:18:21.124566 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.124521    2567 generic.go:356] "Generic (PLEG): container finished" podID="024788f8-bb9c-498b-a729-65a43652b798" containerID="06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2" exitCode=0
Jul 21 21:18:21.124697 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.124573    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"
Jul 21 21:18:21.124697 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.124598    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2" event={"ID":"024788f8-bb9c-498b-a729-65a43652b798","Type":"ContainerDied","Data":"06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2"}
Jul 21 21:18:21.124697 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.124652    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2" event={"ID":"024788f8-bb9c-498b-a729-65a43652b798","Type":"ContainerDied","Data":"091d040918ed271be966fc72623d7684b0372e298a742a93239afae1740fab6a"}
Jul 21 21:18:21.124697 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.124675    2567 scope.go:117] "RemoveContainer" containerID="06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2"
Jul 21 21:18:21.135700 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.135601    2567 scope.go:117] "RemoveContainer" containerID="06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2"
Jul 21 21:18:21.136181 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:18:21.136151    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2\": container with ID starting with 06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2 not found: ID does not exist" containerID="06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2"
Jul 21 21:18:21.136296 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.136193    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2"} err="failed to get container status \"06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2\": rpc error: code = NotFound desc = could not find container \"06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2\": container with ID starting with 06c0e935e65e8d1c8550c28cd53f1d57eed89e6da8cfb5f79237f3860672bac2 not found: ID does not exist"
Jul 21 21:18:21.194633 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.194602    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plqqt\" (UniqueName: \"kubernetes.io/projected/024788f8-bb9c-498b-a729-65a43652b798-kube-api-access-plqqt\") pod \"024788f8-bb9c-498b-a729-65a43652b798\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") "
Jul 21 21:18:21.194776 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.194677    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024788f8-bb9c-498b-a729-65a43652b798-openshift-service-ca\") pod \"024788f8-bb9c-498b-a729-65a43652b798\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") "
Jul 21 21:18:21.194776 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.194699    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024788f8-bb9c-498b-a729-65a43652b798-maas-api-tls\") pod \"024788f8-bb9c-498b-a729-65a43652b798\" (UID: \"024788f8-bb9c-498b-a729-65a43652b798\") "
Jul 21 21:18:21.195057 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.195035    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/024788f8-bb9c-498b-a729-65a43652b798-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "024788f8-bb9c-498b-a729-65a43652b798" (UID: "024788f8-bb9c-498b-a729-65a43652b798"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:18:21.196700 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.196678    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/024788f8-bb9c-498b-a729-65a43652b798-kube-api-access-plqqt" (OuterVolumeSpecName: "kube-api-access-plqqt") pod "024788f8-bb9c-498b-a729-65a43652b798" (UID: "024788f8-bb9c-498b-a729-65a43652b798"). InnerVolumeSpecName "kube-api-access-plqqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:18:21.196762 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.196721    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/024788f8-bb9c-498b-a729-65a43652b798-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "024788f8-bb9c-498b-a729-65a43652b798" (UID: "024788f8-bb9c-498b-a729-65a43652b798"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:18:21.296092 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.296063    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plqqt\" (UniqueName: \"kubernetes.io/projected/024788f8-bb9c-498b-a729-65a43652b798-kube-api-access-plqqt\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:18:21.296092 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.296092    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/024788f8-bb9c-498b-a729-65a43652b798-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:18:21.296280 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.296106    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/024788f8-bb9c-498b-a729-65a43652b798-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:18:21.441672 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.441645    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"]
Jul 21 21:18:21.445208 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:21.445178    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85e235-66f8f87fc-sppx2"]
Jul 21 21:18:23.311273 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:23.311240    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="024788f8-bb9c-498b-a729-65a43652b798" path="/var/lib/kubelet/pods/024788f8-bb9c-498b-a729-65a43652b798/volumes"
Jul 21 21:18:25.494261 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:25.494222    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:25.517435 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:25.517401    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"]
Jul 21 21:18:25.521834 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:25.521805    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-85e235-t864z"]
Jul 21 21:18:26.620268 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:26.620232    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:27.310688 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:27.310652    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="103d7312-d386-4f63-8047-3560d296ae3d" path="/var/lib/kubelet/pods/103d7312-d386-4f63-8047-3560d296ae3d/volumes"
Jul 21 21:18:31.963172 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:31.963105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:32.030759 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.030718    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:32.271619 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.271520    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:32.509580 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.509541    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:32.629960 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.629884    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"]
Jul 21 21:18:32.630268 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.630254    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="024788f8-bb9c-498b-a729-65a43652b798" containerName="maas-api"
Jul 21 21:18:32.630332 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.630269    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="024788f8-bb9c-498b-a729-65a43652b798" containerName="maas-api"
Jul 21 21:18:32.630332 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.630291    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="103d7312-d386-4f63-8047-3560d296ae3d" containerName="revoke-keys"
Jul 21 21:18:32.630332 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.630298    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="103d7312-d386-4f63-8047-3560d296ae3d" containerName="revoke-keys"
Jul 21 21:18:32.630429 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.630348    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="103d7312-d386-4f63-8047-3560d296ae3d" containerName="revoke-keys"
Jul 21 21:18:32.630429 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.630358    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="024788f8-bb9c-498b-a729-65a43652b798" containerName="maas-api"
Jul 21 21:18:32.634746 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.634719    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"
Jul 21 21:18:32.637615 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.637590    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zmths\""
Jul 21 21:18:32.642944 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.642856    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"]
Jul 21 21:18:32.682785 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.682747    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c08f0a9-c1ee-45f7-8443-fe332627b4c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn\" (UID: \"0c08f0a9-c1ee-45f7-8443-fe332627b4c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"
Jul 21 21:18:32.783513 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.783473    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c08f0a9-c1ee-45f7-8443-fe332627b4c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn\" (UID: \"0c08f0a9-c1ee-45f7-8443-fe332627b4c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"
Jul 21 21:18:32.784130 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.784109    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c08f0a9-c1ee-45f7-8443-fe332627b4c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn\" (UID: \"0c08f0a9-c1ee-45f7-8443-fe332627b4c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"
Jul 21 21:18:32.949317 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:32.949282    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"
Jul 21 21:18:33.082097 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:33.082066    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"]
Jul 21 21:18:33.084808 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:18:33.084776    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c08f0a9_c1ee_45f7_8443_fe332627b4c3.slice/crio-0d7004dc233acb4d2bf98b76bbeb29eaecedbbacf1518a44c56f3b888b467ce1 WatchSource:0}: Error finding container 0d7004dc233acb4d2bf98b76bbeb29eaecedbbacf1518a44c56f3b888b467ce1: Status 404 returned error can't find the container with id 0d7004dc233acb4d2bf98b76bbeb29eaecedbbacf1518a44c56f3b888b467ce1
Jul 21 21:18:33.188039 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:33.187981    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn" event={"ID":"0c08f0a9-c1ee-45f7-8443-fe332627b4c3","Type":"ContainerStarted","Data":"0d7004dc233acb4d2bf98b76bbeb29eaecedbbacf1518a44c56f3b888b467ce1"}
Jul 21 21:18:33.395000 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:33.394958    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:33.507645 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:33.507567    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:34.194642 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:34.194612    2567 generic.go:356] "Generic (PLEG): container finished" podID="0c08f0a9-c1ee-45f7-8443-fe332627b4c3" containerID="75409e3ec8e4292519bccff606e26f762320f60d8b5f80dc02b23f1b1d1dc7b1" exitCode=0
Jul 21 21:18:34.195038 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:34.194695    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn" event={"ID":"0c08f0a9-c1ee-45f7-8443-fe332627b4c3","Type":"ContainerDied","Data":"75409e3ec8e4292519bccff606e26f762320f60d8b5f80dc02b23f1b1d1dc7b1"}
Jul 21 21:18:35.349132 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:35.349108    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"
Jul 21 21:18:35.403324 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:35.403288    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c08f0a9-c1ee-45f7-8443-fe332627b4c3-maas-api-service-ca\") pod \"0c08f0a9-c1ee-45f7-8443-fe332627b4c3\" (UID: \"0c08f0a9-c1ee-45f7-8443-fe332627b4c3\") "
Jul 21 21:18:35.403636 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:35.403613    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c08f0a9-c1ee-45f7-8443-fe332627b4c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c08f0a9-c1ee-45f7-8443-fe332627b4c3" (UID: "0c08f0a9-c1ee-45f7-8443-fe332627b4c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:18:35.504295 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:35.504218    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c08f0a9-c1ee-45f7-8443-fe332627b4c3-maas-api-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:18:36.206108 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:36.206078    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"
Jul 21 21:18:36.206278 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:36.206113    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn" event={"ID":"0c08f0a9-c1ee-45f7-8443-fe332627b4c3","Type":"ContainerDied","Data":"0d7004dc233acb4d2bf98b76bbeb29eaecedbbacf1518a44c56f3b888b467ce1"}
Jul 21 21:18:36.206278 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:36.206163    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d7004dc233acb4d2bf98b76bbeb29eaecedbbacf1518a44c56f3b888b467ce1"
Jul 21 21:18:38.289275 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:38.289241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:43.161128 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.161094    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"]
Jul 21 21:18:43.161742 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.161655    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb" podUID="0382fea5-3f00-4995-a1c9-55f3b8057482" containerName="maas-api" containerID="cri-o://81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe" gracePeriod=30
Jul 21 21:18:43.189006 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.188955    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:43.608242 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.608217    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:18:43.676261 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.676220    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0382fea5-3f00-4995-a1c9-55f3b8057482-maas-api-tls\") pod \"0382fea5-3f00-4995-a1c9-55f3b8057482\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") "
Jul 21 21:18:43.676429 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.676312    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrhjl\" (UniqueName: \"kubernetes.io/projected/0382fea5-3f00-4995-a1c9-55f3b8057482-kube-api-access-zrhjl\") pod \"0382fea5-3f00-4995-a1c9-55f3b8057482\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") "
Jul 21 21:18:43.676429 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.676391    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0382fea5-3f00-4995-a1c9-55f3b8057482-openshift-service-ca\") pod \"0382fea5-3f00-4995-a1c9-55f3b8057482\" (UID: \"0382fea5-3f00-4995-a1c9-55f3b8057482\") "
Jul 21 21:18:43.676765 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.676740    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0382fea5-3f00-4995-a1c9-55f3b8057482-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0382fea5-3f00-4995-a1c9-55f3b8057482" (UID: "0382fea5-3f00-4995-a1c9-55f3b8057482"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:18:43.678388 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.678363    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0382fea5-3f00-4995-a1c9-55f3b8057482-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0382fea5-3f00-4995-a1c9-55f3b8057482" (UID: "0382fea5-3f00-4995-a1c9-55f3b8057482"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:18:43.678388 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.678373    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0382fea5-3f00-4995-a1c9-55f3b8057482-kube-api-access-zrhjl" (OuterVolumeSpecName: "kube-api-access-zrhjl") pod "0382fea5-3f00-4995-a1c9-55f3b8057482" (UID: "0382fea5-3f00-4995-a1c9-55f3b8057482"). InnerVolumeSpecName "kube-api-access-zrhjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:18:43.777735 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.777655    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0382fea5-3f00-4995-a1c9-55f3b8057482-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:18:43.777735 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.777685    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrhjl\" (UniqueName: \"kubernetes.io/projected/0382fea5-3f00-4995-a1c9-55f3b8057482-kube-api-access-zrhjl\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:18:43.777735 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:43.777694    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0382fea5-3f00-4995-a1c9-55f3b8057482-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:18:44.249482 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.249443    2567 generic.go:356] "Generic (PLEG): container finished" podID="0382fea5-3f00-4995-a1c9-55f3b8057482" containerID="81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe" exitCode=0
Jul 21 21:18:44.249482 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.249481    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"
Jul 21 21:18:44.250023 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.249529    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb" event={"ID":"0382fea5-3f00-4995-a1c9-55f3b8057482","Type":"ContainerDied","Data":"81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe"}
Jul 21 21:18:44.250023 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.249600    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb" event={"ID":"0382fea5-3f00-4995-a1c9-55f3b8057482","Type":"ContainerDied","Data":"cf6ad732de474020da3f4c7fe02a261a949a0637cdc69bb64eefb1a57fc6ccf9"}
Jul 21 21:18:44.250023 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.249635    2567 scope.go:117] "RemoveContainer" containerID="81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe"
Jul 21 21:18:44.258947 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.258929    2567 scope.go:117] "RemoveContainer" containerID="81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe"
Jul 21 21:18:44.259246 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:18:44.259224    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe\": container with ID starting with 81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe not found: ID does not exist" containerID="81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe"
Jul 21 21:18:44.259305 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.259255    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe"} err="failed to get container status \"81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe\": rpc error: code = NotFound desc = could not find container \"81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe\": container with ID starting with 81114717a95fb3242e6b611a09255019ae13ea6415c6877d7369b312412660fe not found: ID does not exist"
Jul 21 21:18:44.270967 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.270941    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"]
Jul 21 21:18:44.274975 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:44.274950    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c15522-57fcd7687b-tx5wb"]
Jul 21 21:18:45.310791 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:45.310759    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0382fea5-3f00-4995-a1c9-55f3b8057482" path="/var/lib/kubelet/pods/0382fea5-3f00-4995-a1c9-55f3b8057482/volumes"
Jul 21 21:18:47.710565 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:47.710530    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"]
Jul 21 21:18:47.713510 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:47.713482    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-c15522-tt4xn"]
Jul 21 21:18:47.715247 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:47.715214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:48.136948 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:48.136858    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:49.310388 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:49.310347    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c08f0a9-c1ee-45f7-8443-fe332627b4c3" path="/var/lib/kubelet/pods/0c08f0a9-c1ee-45f7-8443-fe332627b4c3/volumes"
Jul 21 21:18:54.232353 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:54.232320    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:54.294330 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:54.294294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:54.516432 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:54.516356    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:54.767837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:54.767745    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:55.144724 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.144639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:55.469673 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.469643    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"]
Jul 21 21:18:55.470133 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.470112    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0382fea5-3f00-4995-a1c9-55f3b8057482" containerName="maas-api"
Jul 21 21:18:55.470178 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.470138    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0382fea5-3f00-4995-a1c9-55f3b8057482" containerName="maas-api"
Jul 21 21:18:55.470178 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.470154    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c08f0a9-c1ee-45f7-8443-fe332627b4c3" containerName="revoke-keys"
Jul 21 21:18:55.470178 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.470163    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c08f0a9-c1ee-45f7-8443-fe332627b4c3" containerName="revoke-keys"
Jul 21 21:18:55.470293 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.470270    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0382fea5-3f00-4995-a1c9-55f3b8057482" containerName="maas-api"
Jul 21 21:18:55.470293 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.470283    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c08f0a9-c1ee-45f7-8443-fe332627b4c3" containerName="revoke-keys"
Jul 21 21:18:55.474838 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.474818    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.478093 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.478075    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f64949-openshift-default-dockercfg-8jzv2\""
Jul 21 21:18:55.491094 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.491068    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"]
Jul 21 21:18:55.492284 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.492258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:55.578640 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578605    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-certs\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.578640 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578642    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-credential-socket\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.578837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578662    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istiod-ca-cert\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.578837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578680    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-envoy\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.578837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578747    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-socket\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.578837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578791    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-token\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.578837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578812    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krtdg\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-kube-api-access-krtdg\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.579021 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-podinfo\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.579021 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.578933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-data\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.679808 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.679780    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-token\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.679967 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.679814    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krtdg\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-kube-api-access-krtdg\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.679967 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.679859    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-podinfo\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.679967 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.679879    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-data\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.679967 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.679903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-certs\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.679967 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.679927    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-credential-socket\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.679967 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.679955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istiod-ca-cert\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.680297 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.679980    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-envoy\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.680297 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.680041    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-socket\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.680459 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.680337    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-data\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.680459 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.680444    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-socket\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.680603 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.680499    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-certs\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.680658 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.680637    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-credential-socket\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.680870 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.680837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istiod-ca-cert\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.682344 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.682298    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-envoy\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.682781 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.682763    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-podinfo\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.690117 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.690098    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-token\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.690246 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.690225    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krtdg\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-kube-api-access-krtdg\") pod \"e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") " pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.713830 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.713802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:55.785949 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.785882    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:55.923042 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.923016    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"]
Jul 21 21:18:55.924968 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:18:55.924941    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570a69b1_73c0_4b1d_86a5_350a19e0aaa9.slice/crio-4793b903c10ea2f601f9b425d4c5e4b290d7c6a4224044039c362dd79dd6b4a3 WatchSource:0}: Error finding container 4793b903c10ea2f601f9b425d4c5e4b290d7c6a4224044039c362dd79dd6b4a3: Status 404 returned error can't find the container with id 4793b903c10ea2f601f9b425d4c5e4b290d7c6a4224044039c362dd79dd6b4a3
Jul 21 21:18:55.927060 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.927028    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:18:55.927165 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.927107    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:18:55.927165 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:55.927149    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:18:56.314329 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:56.314293    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" event={"ID":"570a69b1-73c0-4b1d-86a5-350a19e0aaa9","Type":"ContainerStarted","Data":"37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc"}
Jul 21 21:18:56.314519 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:56.314340    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" event={"ID":"570a69b1-73c0-4b1d-86a5-350a19e0aaa9","Type":"ContainerStarted","Data":"4793b903c10ea2f601f9b425d4c5e4b290d7c6a4224044039c362dd79dd6b4a3"}
Jul 21 21:18:56.335468 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:56.335400    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" podStartSLOduration=1.335380517 podStartE2EDuration="1.335380517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:18:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:18:56.334464069 +0000 UTC m=+2363.544492515" watchObservedRunningTime="2026-07-21 21:18:56.335380517 +0000 UTC m=+2363.545408939"
Jul 21 21:18:56.786904 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:56.786867    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:56.791653 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:56.791629    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:57.319737 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:57.319700    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:57.320548 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:57.320525    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:18:58.329799 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.329763    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:58.418236 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.418204    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"]
Jul 21 21:18:58.422230 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.422211    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.424678 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.424656    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f64949-serving-cert\""
Jul 21 21:18:58.424801 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.424677    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:18:58.424801 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.424674    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:58.431821 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.431797    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"]
Jul 21 21:18:58.501940 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.501898    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:58.506064 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.506041    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.506161 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.506146    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7r28\" (UniqueName: \"kubernetes.io/projected/70aa7c0a-da7f-442f-9c6e-99dcd9137543-kube-api-access-d7r28\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.506215 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.506172    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70aa7c0a-da7f-442f-9c6e-99dcd9137543-openshift-service-ca\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.607586 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.607489    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7r28\" (UniqueName: \"kubernetes.io/projected/70aa7c0a-da7f-442f-9c6e-99dcd9137543-kube-api-access-d7r28\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.607586 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.607530    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70aa7c0a-da7f-442f-9c6e-99dcd9137543-openshift-service-ca\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.607586 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.607573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.607883 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:18:58.607687    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-serving-cert: secret "maas-api-e2e-api-a-f64949-serving-cert" not found
Jul 21 21:18:58.607883 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:18:58.607747    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls podName:70aa7c0a-da7f-442f-9c6e-99dcd9137543 nodeName:}" failed. No retries permitted until 2026-07-21 21:18:59.107730348 +0000 UTC m=+2366.317758762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls") pod "maas-api-e2e-api-a-f64949-76b845795b-qdvph" (UID: "70aa7c0a-da7f-442f-9c6e-99dcd9137543") : secret "maas-api-e2e-api-a-f64949-serving-cert" not found
Jul 21 21:18:58.608344 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.608318    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70aa7c0a-da7f-442f-9c6e-99dcd9137543-openshift-service-ca\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.620024 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.620002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7r28\" (UniqueName: \"kubernetes.io/projected/70aa7c0a-da7f-442f-9c6e-99dcd9137543-kube-api-access-d7r28\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:58.929002 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:58.928958    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:59.112337 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:59.112296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:59.114914 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:59.114886    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls\") pod \"maas-api-e2e-api-a-f64949-76b845795b-qdvph\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:59.169555 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:59.169502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:59.268214 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:59.268131    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:18:59.335398 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:59.335355    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:18:59.487706 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:59.487676    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"]
Jul 21 21:18:59.489659 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:18:59.489624    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70aa7c0a_da7f_442f_9c6e_99dcd9137543.slice/crio-55322ef99d0522b344a3a4aca7e39a217f91ea061b9bf4cbb24838b4b76a9742 WatchSource:0}: Error finding container 55322ef99d0522b344a3a4aca7e39a217f91ea061b9bf4cbb24838b4b76a9742: Status 404 returned error can't find the container with id 55322ef99d0522b344a3a4aca7e39a217f91ea061b9bf4cbb24838b4b76a9742
Jul 21 21:18:59.491018 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:18:59.490998    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 21:19:00.340223 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:00.340130    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph" event={"ID":"70aa7c0a-da7f-442f-9c6e-99dcd9137543","Type":"ContainerStarted","Data":"49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b"}
Jul 21 21:19:00.340223 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:00.340214    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph" event={"ID":"70aa7c0a-da7f-442f-9c6e-99dcd9137543","Type":"ContainerStarted","Data":"55322ef99d0522b344a3a4aca7e39a217f91ea061b9bf4cbb24838b4b76a9742"}
Jul 21 21:19:00.340653 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:00.340333    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:19:00.359838 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:00.359786    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph" podStartSLOduration=1.8510966469999999 podStartE2EDuration="2.359772211s" totalImagesPullingTime="508.675564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:19:00.35656256 +0000 UTC m=+2367.566590982" watchObservedRunningTime="2026-07-21 21:19:00.359772211 +0000 UTC m=+2367.569800632"
Jul 21 21:19:02.883049 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:02.883012    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:03.406267 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:03.406235    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:03.776359 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:03.776319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:03.919715 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:03.919676    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:06.350761 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:06.350731    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:19:07.054224 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.054187    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:07.171169 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.171137    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:07.171536 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.171512    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"]
Jul 21 21:19:07.175248 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.175228    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.180370 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.180345    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-60b004-serving-cert\""
Jul 21 21:19:07.182969 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.182947    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"]
Jul 21 21:19:07.263943 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.263895    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:07.285871 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.285840    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.286081 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.285894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d82491a-876c-4b3e-90a2-0e0676727061-openshift-service-ca\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.286081 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.286036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4dfz\" (UniqueName: \"kubernetes.io/projected/0d82491a-876c-4b3e-90a2-0e0676727061-kube-api-access-m4dfz\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.387549 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.387461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.388032 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.387710    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d82491a-876c-4b3e-90a2-0e0676727061-openshift-service-ca\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.388032 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:19:07.387769    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-serving-cert: secret "maas-api-e2e-api-b-60b004-serving-cert" not found
Jul 21 21:19:07.388032 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:19:07.387841    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls podName:0d82491a-876c-4b3e-90a2-0e0676727061 nodeName:}" failed. No retries permitted until 2026-07-21 21:19:07.887819605 +0000 UTC m=+2375.097848018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls") pod "maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x" (UID: "0d82491a-876c-4b3e-90a2-0e0676727061") : secret "maas-api-e2e-api-b-60b004-serving-cert" not found
Jul 21 21:19:07.388222 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.388026    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4dfz\" (UniqueName: \"kubernetes.io/projected/0d82491a-876c-4b3e-90a2-0e0676727061-kube-api-access-m4dfz\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.388524 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.388500    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d82491a-876c-4b3e-90a2-0e0676727061-openshift-service-ca\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.398070 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.398040    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4dfz\" (UniqueName: \"kubernetes.io/projected/0d82491a-876c-4b3e-90a2-0e0676727061-kube-api-access-m4dfz\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.632122 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.632083    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:07.893236 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.893191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.895630 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.895608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls\") pod \"maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:07.914825 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:07.914771    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:08.088010 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:08.087944    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:08.096263 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:08.096226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:08.266554 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:08.266459    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"]
Jul 21 21:19:08.272222 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:19:08.272192    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d82491a_876c_4b3e_90a2_0e0676727061.slice/crio-30627f1b66b0f24319c91fc3372aa1db89d3b1ac468e47e7a976230f723e601f WatchSource:0}: Error finding container 30627f1b66b0f24319c91fc3372aa1db89d3b1ac468e47e7a976230f723e601f: Status 404 returned error can't find the container with id 30627f1b66b0f24319c91fc3372aa1db89d3b1ac468e47e7a976230f723e601f
Jul 21 21:19:08.386495 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:08.386460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x" event={"ID":"0d82491a-876c-4b3e-90a2-0e0676727061","Type":"ContainerStarted","Data":"30627f1b66b0f24319c91fc3372aa1db89d3b1ac468e47e7a976230f723e601f"}
Jul 21 21:19:09.393667 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:09.393632    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x" event={"ID":"0d82491a-876c-4b3e-90a2-0e0676727061","Type":"ContainerStarted","Data":"c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad"}
Jul 21 21:19:09.394104 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:09.393696    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:09.411729 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:09.411685    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x" podStartSLOduration=2.053954942 podStartE2EDuration="2.411670721s" totalImagesPullingTime="357.715779ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:19:09.410467869 +0000 UTC m=+2376.620496291" watchObservedRunningTime="2026-07-21 21:19:09.411670721 +0000 UTC m=+2376.621699142"
Jul 21 21:19:11.567932 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:11.567891    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:13.362172 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:13.362121    2567 patch_prober.go:28] interesting pod/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 21 21:19:13.362710 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:13.362185    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 21 21:19:14.421843 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:14.421808    2567 generic.go:356] "Generic (PLEG): container finished" podID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerID="37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc" exitCode=0
Jul 21 21:19:14.422292 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:14.421887    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" event={"ID":"570a69b1-73c0-4b1d-86a5-350a19e0aaa9","Type":"ContainerDied","Data":"37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc"}
Jul 21 21:19:14.422353 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:14.422304    2567 scope.go:117] "RemoveContainer" containerID="37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc"
Jul 21 21:19:14.423247 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:14.423222    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:19:14.423325 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:14.423286    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:19:14.423371 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:14.423331    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:19:15.404423 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:15.404390    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:19:15.429713 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:15.429674    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" event={"ID":"570a69b1-73c0-4b1d-86a5-350a19e0aaa9","Type":"ContainerStarted","Data":"9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21"}
Jul 21 21:19:15.787097 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:15.787046    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:19:15.787290 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:15.787247    2567 patch_prober.go:28] interesting pod/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 21 21:19:15.787368 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:15.787313    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 21 21:19:16.786603 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:16.786565    2567 patch_prober.go:28] interesting pod/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 21 21:19:16.787028 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:16.786628    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 21 21:19:17.786755 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:17.786715    2567 patch_prober.go:28] interesting pod/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 21 21:19:17.787273 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:17.786802    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 21 21:19:18.787065 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:18.787026    2567 patch_prober.go:28] interesting pod/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 21 21:19:18.787492 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:18.787091    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 21 21:19:19.789681 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:19.789647    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:19:19.790082 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:19.790015    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:19:19.790808 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:19.790786    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:19:20.093764 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:20.093645    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:20.194795 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:20.194748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:29.003247 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:29.003194    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:29.152554 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:29.152484    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:30.478576 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:30.478537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:30.579416 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:30.579374    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:33.466018 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:33.465975    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:19:33.474110 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:33.474088    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:19:35.726236 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:35.726199    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"]
Jul 21 21:19:35.728026 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:35.727965    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph" podUID="70aa7c0a-da7f-442f-9c6e-99dcd9137543" containerName="maas-api" containerID="cri-o://49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b" gracePeriod=30
Jul 21 21:19:35.757515 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:35.757487    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:35.997498 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:35.997470    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:19:36.044469 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.044440    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70aa7c0a-da7f-442f-9c6e-99dcd9137543-openshift-service-ca\") pod \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") "
Jul 21 21:19:36.044632 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.044487    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7r28\" (UniqueName: \"kubernetes.io/projected/70aa7c0a-da7f-442f-9c6e-99dcd9137543-kube-api-access-d7r28\") pod \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") "
Jul 21 21:19:36.044632 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.044531    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls\") pod \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\" (UID: \"70aa7c0a-da7f-442f-9c6e-99dcd9137543\") "
Jul 21 21:19:36.044846 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.044812    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70aa7c0a-da7f-442f-9c6e-99dcd9137543-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "70aa7c0a-da7f-442f-9c6e-99dcd9137543" (UID: "70aa7c0a-da7f-442f-9c6e-99dcd9137543"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:19:36.046603 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.046573    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "70aa7c0a-da7f-442f-9c6e-99dcd9137543" (UID: "70aa7c0a-da7f-442f-9c6e-99dcd9137543"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:19:36.046704 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.046623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70aa7c0a-da7f-442f-9c6e-99dcd9137543-kube-api-access-d7r28" (OuterVolumeSpecName: "kube-api-access-d7r28") pod "70aa7c0a-da7f-442f-9c6e-99dcd9137543" (UID: "70aa7c0a-da7f-442f-9c6e-99dcd9137543"). InnerVolumeSpecName "kube-api-access-d7r28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:19:36.146037 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.146011    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70aa7c0a-da7f-442f-9c6e-99dcd9137543-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:36.146037 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.146033    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7r28\" (UniqueName: \"kubernetes.io/projected/70aa7c0a-da7f-442f-9c6e-99dcd9137543-kube-api-access-d7r28\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:36.146190 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.146043    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70aa7c0a-da7f-442f-9c6e-99dcd9137543-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:36.555409 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.555377    2567 generic.go:356] "Generic (PLEG): container finished" podID="70aa7c0a-da7f-442f-9c6e-99dcd9137543" containerID="49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b" exitCode=0
Jul 21 21:19:36.555566 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.555420    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"
Jul 21 21:19:36.555566 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.555462    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph" event={"ID":"70aa7c0a-da7f-442f-9c6e-99dcd9137543","Type":"ContainerDied","Data":"49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b"}
Jul 21 21:19:36.555566 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.555511    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph" event={"ID":"70aa7c0a-da7f-442f-9c6e-99dcd9137543","Type":"ContainerDied","Data":"55322ef99d0522b344a3a4aca7e39a217f91ea061b9bf4cbb24838b4b76a9742"}
Jul 21 21:19:36.555566 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.555533    2567 scope.go:117] "RemoveContainer" containerID="49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b"
Jul 21 21:19:36.567760 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.567746    2567 scope.go:117] "RemoveContainer" containerID="49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b"
Jul 21 21:19:36.568021 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:19:36.568004    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b\": container with ID starting with 49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b not found: ID does not exist" containerID="49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b"
Jul 21 21:19:36.568083 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.568028    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b"} err="failed to get container status \"49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b\": rpc error: code = NotFound desc = could not find container \"49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b\": container with ID starting with 49ebf1fd2d10d316642a6d407f953ae8d7bc36930cd3f57d3cb5a6375a20ff8b not found: ID does not exist"
Jul 21 21:19:36.590188 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.590163    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"]
Jul 21 21:19:36.603571 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:36.603548    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f64949-76b845795b-qdvph"]
Jul 21 21:19:37.311189 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:37.311152    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70aa7c0a-da7f-442f-9c6e-99dcd9137543" path="/var/lib/kubelet/pods/70aa7c0a-da7f-442f-9c6e-99dcd9137543/volumes"
Jul 21 21:19:40.000619 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:40.000587    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:40.408293 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:40.408203    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:46.546376 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:46.546341    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:46.644184 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:46.644144    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:46.804243 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:46.804154    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"]
Jul 21 21:19:46.804415 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:46.804393    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy" containerID="cri-o://9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21" gracePeriod=30
Jul 21 21:19:46.824966 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:46.824938    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:47.087798 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:47.087713    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:47.893269 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:47.893223    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:47.974580 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:47.974547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:48.806807 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:48.806761    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:48.912388 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:48.912349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:49.793070 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:49.792976    2567 patch_prober.go:28] interesting pod/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 21:19:49.793070 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:49.793062    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 21:19:52.066641 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.066590    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:19:52.192922 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.192890    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-envoy\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.192922 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.192927    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-credential-socket\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.193199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.192974    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-certs\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.193199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193023    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-podinfo\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.193199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193056    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krtdg\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-kube-api-access-krtdg\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.193199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193077    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-data\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.193199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193107    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istiod-ca-cert\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.193199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193131    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-token\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.193199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193181    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-socket\") pod \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\" (UID: \"570a69b1-73c0-4b1d-86a5-350a19e0aaa9\") "
Jul 21 21:19:52.193574 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193338    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:19:52.193574 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193373    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:19:52.193574 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193498    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-credential-socket\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.193574 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193519    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-certs\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.193787 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193579    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:19:52.193850 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.193784    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:19:52.194239 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.194216    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-data" (OuterVolumeSpecName: "istio-data") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:19:52.195381 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.195360    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-token" (OuterVolumeSpecName: "istio-token") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:19:52.195469 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.195359    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 21:19:52.195620 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.195595    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-kube-api-access-krtdg" (OuterVolumeSpecName: "kube-api-access-krtdg") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "kube-api-access-krtdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:19:52.195620 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.195603    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "570a69b1-73c0-4b1d-86a5-350a19e0aaa9" (UID: "570a69b1-73c0-4b1d-86a5-350a19e0aaa9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:19:52.294325 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.294291    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-podinfo\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.294325 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.294318    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krtdg\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-kube-api-access-krtdg\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.294325 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.294327    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-data\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.294546 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.294337    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istiod-ca-cert\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.294546 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.294346    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-token\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.294546 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.294355    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-workload-socket\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.294546 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.294363    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/570a69b1-73c0-4b1d-86a5-350a19e0aaa9-istio-envoy\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:19:52.649763 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.649730    2567 generic.go:356] "Generic (PLEG): container finished" podID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerID="9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21" exitCode=0
Jul 21 21:19:52.649965 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.649786    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"
Jul 21 21:19:52.649965 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.649781    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" event={"ID":"570a69b1-73c0-4b1d-86a5-350a19e0aaa9","Type":"ContainerDied","Data":"9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21"}
Jul 21 21:19:52.649965 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.649921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm" event={"ID":"570a69b1-73c0-4b1d-86a5-350a19e0aaa9","Type":"ContainerDied","Data":"4793b903c10ea2f601f9b425d4c5e4b290d7c6a4224044039c362dd79dd6b4a3"}
Jul 21 21:19:52.649965 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.649948    2567 scope.go:117] "RemoveContainer" containerID="9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21"
Jul 21 21:19:52.659770 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.659755    2567 scope.go:117] "RemoveContainer" containerID="37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc"
Jul 21 21:19:52.667542 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.667524    2567 scope.go:117] "RemoveContainer" containerID="9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21"
Jul 21 21:19:52.667771 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:19:52.667752    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21\": container with ID starting with 9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21 not found: ID does not exist" containerID="9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21"
Jul 21 21:19:52.667837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.667779    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21"} err="failed to get container status \"9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21\": rpc error: code = NotFound desc = could not find container \"9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21\": container with ID starting with 9abfcb6dd6cc53815a9e6873efdacbc705a142419d76a56bc208a67198d03c21 not found: ID does not exist"
Jul 21 21:19:52.667837 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.667798    2567 scope.go:117] "RemoveContainer" containerID="37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc"
Jul 21 21:19:52.668045 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:19:52.668027    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc\": container with ID starting with 37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc not found: ID does not exist" containerID="37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc"
Jul 21 21:19:52.668123 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.668049    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc"} err="failed to get container status \"37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc\": rpc error: code = NotFound desc = could not find container \"37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc\": container with ID starting with 37e02998bf03ad5fc79043cd66c31dc6794601b3508231477f1cdce9b18d68fc not found: ID does not exist"
Jul 21 21:19:52.681805 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.681783    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"]
Jul 21 21:19:52.688146 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:52.688123    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f64949-openshift-default-5b4b6964d7-m5hzm"]
Jul 21 21:19:53.312319 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:53.312283    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" path="/var/lib/kubelet/pods/570a69b1-73c0-4b1d-86a5-350a19e0aaa9/volumes"
Jul 21 21:19:57.520730 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:57.520640    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:19:57.597134 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:19:57.597095    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:02.887254 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:02.887217    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"]
Jul 21 21:20:02.887725 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:02.887516    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x" podUID="0d82491a-876c-4b3e-90a2-0e0676727061" containerName="maas-api" containerID="cri-o://c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad" gracePeriod=30
Jul 21 21:20:02.927323 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:02.927289    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:03.129549 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.129524    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:20:03.287786 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.287750    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls\") pod \"0d82491a-876c-4b3e-90a2-0e0676727061\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") "
Jul 21 21:20:03.287978 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.287817    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4dfz\" (UniqueName: \"kubernetes.io/projected/0d82491a-876c-4b3e-90a2-0e0676727061-kube-api-access-m4dfz\") pod \"0d82491a-876c-4b3e-90a2-0e0676727061\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") "
Jul 21 21:20:03.287978 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.287880    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d82491a-876c-4b3e-90a2-0e0676727061-openshift-service-ca\") pod \"0d82491a-876c-4b3e-90a2-0e0676727061\" (UID: \"0d82491a-876c-4b3e-90a2-0e0676727061\") "
Jul 21 21:20:03.288269 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.288243    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d82491a-876c-4b3e-90a2-0e0676727061-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d82491a-876c-4b3e-90a2-0e0676727061" (UID: "0d82491a-876c-4b3e-90a2-0e0676727061"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:20:03.289901 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.289880    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d82491a-876c-4b3e-90a2-0e0676727061" (UID: "0d82491a-876c-4b3e-90a2-0e0676727061"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:20:03.290006 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.289973    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d82491a-876c-4b3e-90a2-0e0676727061-kube-api-access-m4dfz" (OuterVolumeSpecName: "kube-api-access-m4dfz") pod "0d82491a-876c-4b3e-90a2-0e0676727061" (UID: "0d82491a-876c-4b3e-90a2-0e0676727061"). InnerVolumeSpecName "kube-api-access-m4dfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:20:03.389604 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.389567    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d82491a-876c-4b3e-90a2-0e0676727061-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:20:03.389604 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.389598    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d82491a-876c-4b3e-90a2-0e0676727061-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:20:03.389778 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.389611    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4dfz\" (UniqueName: \"kubernetes.io/projected/0d82491a-876c-4b3e-90a2-0e0676727061-kube-api-access-m4dfz\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:20:03.708741 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.708707    2567 generic.go:356] "Generic (PLEG): container finished" podID="0d82491a-876c-4b3e-90a2-0e0676727061" containerID="c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad" exitCode=0
Jul 21 21:20:03.708948 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.708749    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"
Jul 21 21:20:03.708948 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.708788    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x" event={"ID":"0d82491a-876c-4b3e-90a2-0e0676727061","Type":"ContainerDied","Data":"c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad"}
Jul 21 21:20:03.708948 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.708840    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x" event={"ID":"0d82491a-876c-4b3e-90a2-0e0676727061","Type":"ContainerDied","Data":"30627f1b66b0f24319c91fc3372aa1db89d3b1ac468e47e7a976230f723e601f"}
Jul 21 21:20:03.708948 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.708861    2567 scope.go:117] "RemoveContainer" containerID="c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad"
Jul 21 21:20:03.719885 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.719869    2567 scope.go:117] "RemoveContainer" containerID="c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad"
Jul 21 21:20:03.720194 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:20:03.720173    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad\": container with ID starting with c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad not found: ID does not exist" containerID="c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad"
Jul 21 21:20:03.720251 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.720203    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad"} err="failed to get container status \"c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad\": rpc error: code = NotFound desc = could not find container \"c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad\": container with ID starting with c9e5642ba522b551fb858075218d3c24cec7aac5d74edb5aee3275118d31f2ad not found: ID does not exist"
Jul 21 21:20:03.727571 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.727548    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"]
Jul 21 21:20:03.732328 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:03.732309    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-60b004-7d4ff4dcdc-pvx7x"]
Jul 21 21:20:05.310439 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:05.310406    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d82491a-876c-4b3e-90a2-0e0676727061" path="/var/lib/kubelet/pods/0d82491a-876c-4b3e-90a2-0e0676727061/volumes"
Jul 21 21:20:07.297141 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:07.297104    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:07.725323 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:07.725283    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:14.736555 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:14.736519    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:14.874672 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:14.874637    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:15.112242 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:15.112144    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:15.424958 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:15.424924    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:15.837320 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:15.837224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:16.351843 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:16.351794    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:16.889196 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:16.889151    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:19.924039 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:19.923977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:20.044841 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.044809    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"]
Jul 21 21:20:20.045180 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045166    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d82491a-876c-4b3e-90a2-0e0676727061" containerName="maas-api"
Jul 21 21:20:20.045238 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045182    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d82491a-876c-4b3e-90a2-0e0676727061" containerName="maas-api"
Jul 21 21:20:20.045238 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045194    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70aa7c0a-da7f-442f-9c6e-99dcd9137543" containerName="maas-api"
Jul 21 21:20:20.045238 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045200    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="70aa7c0a-da7f-442f-9c6e-99dcd9137543" containerName="maas-api"
Jul 21 21:20:20.045238 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045209    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy"
Jul 21 21:20:20.045238 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045214    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy"
Jul 21 21:20:20.045427 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045283    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy"
Jul 21 21:20:20.045427 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045292    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="70aa7c0a-da7f-442f-9c6e-99dcd9137543" containerName="maas-api"
Jul 21 21:20:20.045427 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045300    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy"
Jul 21 21:20:20.045427 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.045306    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d82491a-876c-4b3e-90a2-0e0676727061" containerName="maas-api"
Jul 21 21:20:20.048427 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.048408    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.057169 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:20:20.057134    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" podUID="689cbd30-6f15-4603-b386-a8c7fa0a211b" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.057293 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:20:20.057177    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-shared-a-0c5ebd-serving-cert\" is forbidden: User \"system:node:ip-10-0-130-177.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-130-177.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0c5ebd-serving-cert\"" type="*v1.Secret"
Jul 21 21:20:20.059566 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.059548    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-glxzb\""
Jul 21 21:20:20.093818 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.093793    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"]
Jul 21 21:20:20.135667 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.135639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.135863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.135714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/689cbd30-6f15-4603-b386-a8c7fa0a211b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.135863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.135797    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6k8h\" (UniqueName: \"kubernetes.io/projected/689cbd30-6f15-4603-b386-a8c7fa0a211b-kube-api-access-g6k8h\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.230002 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.229892    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:20.237188 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.237150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.237331 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.237271    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/689cbd30-6f15-4603-b386-a8c7fa0a211b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.237331 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.237310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6k8h\" (UniqueName: \"kubernetes.io/projected/689cbd30-6f15-4603-b386-a8c7fa0a211b-kube-api-access-g6k8h\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.238061 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.238039    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/689cbd30-6f15-4603-b386-a8c7fa0a211b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:20.261327 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:20.261296    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6k8h\" (UniqueName: \"kubernetes.io/projected/689cbd30-6f15-4603-b386-a8c7fa0a211b-kube-api-access-g6k8h\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:21.131508 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:21.131470    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:21.237934 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:20:21.237894    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-serving-cert: failed to sync secret cache: timed out waiting for the condition
Jul 21 21:20:21.238148 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:20:21.238019    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls podName:689cbd30-6f15-4603-b386-a8c7fa0a211b nodeName:}" failed. No retries permitted until 2026-07-21 21:20:21.73797721 +0000 UTC m=+2448.948005621 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls") pod "maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5" (UID: "689cbd30-6f15-4603-b386-a8c7fa0a211b") : failed to sync secret cache: timed out waiting for the condition
Jul 21 21:20:21.362970 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:21.362939    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-0c5ebd-serving-cert\""
Jul 21 21:20:21.649601 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:21.649552    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:21.750444 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:21.750408    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:21.752808 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:21.752773    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls\") pod \"maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:21.805564 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:21.805533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:21.858943 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:21.858912    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:22.010373 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:22.010253    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"]
Jul 21 21:20:22.012541 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:20:22.012514    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod689cbd30_6f15_4603_b386_a8c7fa0a211b.slice/crio-5b63f29c4b80ba4570557c9b17c41b1ccae454ab2a064fd8c85491c7a7465dcf WatchSource:0}: Error finding container 5b63f29c4b80ba4570557c9b17c41b1ccae454ab2a064fd8c85491c7a7465dcf: Status 404 returned error can't find the container with id 5b63f29c4b80ba4570557c9b17c41b1ccae454ab2a064fd8c85491c7a7465dcf
Jul 21 21:20:22.814591 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:22.814549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5" event={"ID":"689cbd30-6f15-4603-b386-a8c7fa0a211b","Type":"ContainerStarted","Data":"9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb"}
Jul 21 21:20:22.814970 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:22.814614    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5" event={"ID":"689cbd30-6f15-4603-b386-a8c7fa0a211b","Type":"ContainerStarted","Data":"5b63f29c4b80ba4570557c9b17c41b1ccae454ab2a064fd8c85491c7a7465dcf"}
Jul 21 21:20:22.814970 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:22.814721    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:22.856285 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:22.856232    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5" podStartSLOduration=3.476328101 podStartE2EDuration="3.856216896s" totalImagesPullingTime="379.888795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:20:22.85252872 +0000 UTC m=+2450.062557160" watchObservedRunningTime="2026-07-21 21:20:22.856216896 +0000 UTC m=+2450.066245346"
Jul 21 21:20:24.183838 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.183805    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:24.533377 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.533280    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:24.949426 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.949393    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"]
Jul 21 21:20:24.949755 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.949742    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy"
Jul 21 21:20:24.949804 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.949756    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="570a69b1-73c0-4b1d-86a5-350a19e0aaa9" containerName="istio-proxy"
Jul 21 21:20:24.952902 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.952884    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.956451 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.956428    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-80e9ec-openshift-default-dockercfg-jgsmk\""
Jul 21 21:20:24.980350 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-data\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.980461 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-credential-socket\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.980461 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-certs\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.980461 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980405    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-token\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.980461 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980435    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-envoy\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.980738 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980545    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-podinfo\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.980738 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-socket\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.980738 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b42f7eb1-15cf-46e6-89c4-1abe81167120-istiod-ca-cert\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.980738 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.980675    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq4pn\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-kube-api-access-sq4pn\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:24.990526 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.990489    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"]
Jul 21 21:20:24.999116 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:24.999076    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:25.081639 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-data\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.081834 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081648    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-credential-socket\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.081834 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081670    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-certs\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.081834 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-token\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.081834 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081737    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-envoy\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.081834 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-podinfo\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.081834 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081809    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-socket\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.082184 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081853    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b42f7eb1-15cf-46e6-89c4-1abe81167120-istiod-ca-cert\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.082184 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.081875    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq4pn\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-kube-api-access-sq4pn\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.082184 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.082089    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-credential-socket\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.082184 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.082138    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-certs\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.082346 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.082200    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-data\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.082397 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.082373    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-socket\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.082651 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.082635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b42f7eb1-15cf-46e6-89c4-1abe81167120-istiod-ca-cert\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.084326 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.084304    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-envoy\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.084326 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.084310    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-podinfo\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.101913 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.101883    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq4pn\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-kube-api-access-sq4pn\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.104961 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.104943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-token\") pod \"e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") " pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.263226 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.263128    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:25.282662 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.282629    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:25.424189 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.424162    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"]
Jul 21 21:20:25.425962 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:20:25.425933    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42f7eb1_15cf_46e6_89c4_1abe81167120.slice/crio-a59a0ffac207e1414e07d659f63a51572d09cb252659c3eaf05d238a1768581c WatchSource:0}: Error finding container a59a0ffac207e1414e07d659f63a51572d09cb252659c3eaf05d238a1768581c: Status 404 returned error can't find the container with id a59a0ffac207e1414e07d659f63a51572d09cb252659c3eaf05d238a1768581c
Jul 21 21:20:25.428129 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.428102    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:20:25.428198 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.428163    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:20:25.428198 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.428190    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:20:25.831570 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.831531    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" event={"ID":"b42f7eb1-15cf-46e6-89c4-1abe81167120","Type":"ContainerStarted","Data":"a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790"}
Jul 21 21:20:25.831791 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.831626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" event={"ID":"b42f7eb1-15cf-46e6-89c4-1abe81167120","Type":"ContainerStarted","Data":"a59a0ffac207e1414e07d659f63a51572d09cb252659c3eaf05d238a1768581c"}
Jul 21 21:20:25.871852 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:25.871796    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" podStartSLOduration=1.871782229 podStartE2EDuration="1.871782229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:20:25.870569985 +0000 UTC m=+2453.080598459" watchObservedRunningTime="2026-07-21 21:20:25.871782229 +0000 UTC m=+2453.081810649"
Jul 21 21:20:26.264383 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:26.264345    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:26.269140 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:26.269114    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:26.837604 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:26.837569    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:26.838725 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:26.838700    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:28.076293 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.076258    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:28.160356 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.160322    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"]
Jul 21 21:20:28.164325 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.164308    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.169941 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.169916    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-80e9ec-serving-cert\""
Jul 21 21:20:28.181462 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.181435    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"]
Jul 21 21:20:28.206360 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.206333    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:28.210483 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.210461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnc7x\" (UniqueName: \"kubernetes.io/projected/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-kube-api-access-qnc7x\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.210561 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.210504    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.210561 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.210525    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.311793 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.311764    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.312014 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.311803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.312014 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.311898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnc7x\" (UniqueName: \"kubernetes.io/projected/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-kube-api-access-qnc7x\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.312014 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:20:28.311970    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-serving-cert: secret "maas-api-e2e-shared-b-80e9ec-serving-cert" not found
Jul 21 21:20:28.312217 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:20:28.312058    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls podName:bec2a8ad-4c2e-401e-8e01-900bd95cb5cc nodeName:}" failed. No retries permitted until 2026-07-21 21:20:28.812043642 +0000 UTC m=+2456.022072041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls") pod "maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx" (UID: "bec2a8ad-4c2e-401e-8e01-900bd95cb5cc") : secret "maas-api-e2e-shared-b-80e9ec-serving-cert" not found
Jul 21 21:20:28.312424 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.312406    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.330297 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.330234    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnc7x\" (UniqueName: \"kubernetes.io/projected/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-kube-api-access-qnc7x\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.395412 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.395377    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:28.517447 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.517407    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:28.816259 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.816221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.818551 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.818528    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls\") pod \"maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:28.825792 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.825770    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:20:28.945945 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:28.945903    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:29.043598 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:29.043550    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:29.075142 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:29.075073    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:29.263429 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:29.263401    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"]
Jul 21 21:20:29.264176 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:20:29.264152    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec2a8ad_4c2e_401e_8e01_900bd95cb5cc.slice/crio-ec0b5f79dcf22fddc7974479c19abe6631e8e5c535e14b877891bba2a3d530bd WatchSource:0}: Error finding container ec0b5f79dcf22fddc7974479c19abe6631e8e5c535e14b877891bba2a3d530bd: Status 404 returned error can't find the container with id ec0b5f79dcf22fddc7974479c19abe6631e8e5c535e14b877891bba2a3d530bd
Jul 21 21:20:29.861855 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:29.861819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx" event={"ID":"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc","Type":"ContainerStarted","Data":"2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239"}
Jul 21 21:20:29.862023 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:29.862009    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:29.862071 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:29.862037    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx" event={"ID":"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc","Type":"ContainerStarted","Data":"ec0b5f79dcf22fddc7974479c19abe6631e8e5c535e14b877891bba2a3d530bd"}
Jul 21 21:20:29.886365 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:29.886322    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx" podStartSLOduration=1.425333605 podStartE2EDuration="1.886306464s" totalImagesPullingTime="460.972859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 21:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:20:29.885408179 +0000 UTC m=+2457.095436599" watchObservedRunningTime="2026-07-21 21:20:29.886306464 +0000 UTC m=+2457.096334886"
Jul 21 21:20:32.571446 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:32.571411    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:35.872008 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:35.871951    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:20:40.813374 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:40.813337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:41.839578 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:41.839494    2567 patch_prober.go:28] interesting pod/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 21 21:20:41.839578 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:41.839555    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 21 21:20:41.925611 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:41.925581    2567 generic.go:356] "Generic (PLEG): container finished" podID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerID="a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790" exitCode=0
Jul 21 21:20:41.925748 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:41.925636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" event={"ID":"b42f7eb1-15cf-46e6-89c4-1abe81167120","Type":"ContainerDied","Data":"a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790"}
Jul 21 21:20:41.926069 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:41.926052    2567 scope.go:117] "RemoveContainer" containerID="a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790"
Jul 21 21:20:41.931000 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:41.927330    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:20:41.931000 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:41.927507    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:20:41.931000 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:41.927604    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 21 21:20:42.646820 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:42.646783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:42.745116 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:42.745075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:42.933831 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:42.933794    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" event={"ID":"b42f7eb1-15cf-46e6-89c4-1abe81167120","Type":"ContainerStarted","Data":"973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c"}
Jul 21 21:20:43.264253 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:43.264096    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:43.264435 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:43.264398    2567 patch_prober.go:28] interesting pod/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 21 21:20:43.264512 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:43.264452    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 21 21:20:44.264658 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:44.264619    2567 patch_prober.go:28] interesting pod/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 21 21:20:44.265128 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:44.264689    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 21 21:20:45.264171 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:45.264128    2567 patch_prober.go:28] interesting pod/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 21 21:20:45.264380 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:45.264198    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 21 21:20:46.264506 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:46.264468    2567 patch_prober.go:28] interesting pod/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Jul 21 21:20:46.265027 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:46.264528    2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Jul 21 21:20:47.267283 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:47.267244    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:47.267657 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:47.267496    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:47.268157 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:47.268142    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:20:49.826273 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:49.826234    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:20:49.931199 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:20:49.931160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:21:11.059560 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:21:11.059516    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:23:15.262870 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:23:15.262786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:23:16.131214 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:23:16.131179    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:23:26.247479 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:23:26.247441    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:23:40.277382 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:23:40.277342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:23:57.408452 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:23:57.408407    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:24:33.502712 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:33.502684    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:24:33.509872 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:33.509851    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-pv66t_9c6e732b-dd0c-4b2e-92c3-28daefa0db18/ovn-acl-logging/0.log"
Jul 21 21:24:33.664097 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:33.664060    2567 scope.go:117] "RemoveContainer" containerID="75409e3ec8e4292519bccff606e26f762320f60d8b5f80dc02b23f1b1d1dc7b1"
Jul 21 21:24:33.674738 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:33.674716    2567 scope.go:117] "RemoveContainer" containerID="01bb80a61261b7f2595be5701c17359e592957e2a71880d5d8ec0502defdcbe8"
Jul 21 21:24:35.335840 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:35.335801    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:24:36.311613 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:36.311580    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:24:45.003884 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:45.003847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:24:48.790926 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:48.790897    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:24:48.892292 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:48.892257    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:24:56.131868 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:56.131833    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:24:56.972702 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:24:56.972667    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:00.110090 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:00.110052    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:02.570806 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:02.570772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:23.398298 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:23.398260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:23.456246 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:23.456203    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:43.468379 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:43.468347    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:43.565885 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:43.565851    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:50.491975 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:50.491931    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:50.618122 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:50.618093    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:50.722787 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:50.722752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:50.818507 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:50.818419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:25:51.424286 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:25:51.424254    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:11.082435 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:11.082352    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:11.182127 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:11.182092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:13.508296 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:13.508259    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:13.564391 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:13.564362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:20.980101 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:20.980068    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"]
Jul 21 21:26:20.980746 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:20.980691    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5" podUID="689cbd30-6f15-4603-b386-a8c7fa0a211b" containerName="maas-api" containerID="cri-o://9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb" gracePeriod=30
Jul 21 21:26:21.010959 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.010932    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:21.229978 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.229953    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:26:21.310666 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.310585    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/689cbd30-6f15-4603-b386-a8c7fa0a211b-openshift-service-ca\") pod \"689cbd30-6f15-4603-b386-a8c7fa0a211b\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") "
Jul 21 21:26:21.310666 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.310649    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls\") pod \"689cbd30-6f15-4603-b386-a8c7fa0a211b\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") "
Jul 21 21:26:21.310854 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.310681    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6k8h\" (UniqueName: \"kubernetes.io/projected/689cbd30-6f15-4603-b386-a8c7fa0a211b-kube-api-access-g6k8h\") pod \"689cbd30-6f15-4603-b386-a8c7fa0a211b\" (UID: \"689cbd30-6f15-4603-b386-a8c7fa0a211b\") "
Jul 21 21:26:21.311023 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.310980    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/689cbd30-6f15-4603-b386-a8c7fa0a211b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "689cbd30-6f15-4603-b386-a8c7fa0a211b" (UID: "689cbd30-6f15-4603-b386-a8c7fa0a211b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:26:21.312753 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.312731    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/689cbd30-6f15-4603-b386-a8c7fa0a211b-kube-api-access-g6k8h" (OuterVolumeSpecName: "kube-api-access-g6k8h") pod "689cbd30-6f15-4603-b386-a8c7fa0a211b" (UID: "689cbd30-6f15-4603-b386-a8c7fa0a211b"). InnerVolumeSpecName "kube-api-access-g6k8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:26:21.312926 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.312909    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "689cbd30-6f15-4603-b386-a8c7fa0a211b" (UID: "689cbd30-6f15-4603-b386-a8c7fa0a211b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:26:21.411339 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.411304    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/689cbd30-6f15-4603-b386-a8c7fa0a211b-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:21.411339 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.411335    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6k8h\" (UniqueName: \"kubernetes.io/projected/689cbd30-6f15-4603-b386-a8c7fa0a211b-kube-api-access-g6k8h\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:21.411339 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.411349    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/689cbd30-6f15-4603-b386-a8c7fa0a211b-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:21.783364 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.783325    2567 generic.go:356] "Generic (PLEG): container finished" podID="689cbd30-6f15-4603-b386-a8c7fa0a211b" containerID="9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb" exitCode=0
Jul 21 21:26:21.783535 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.783375    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"
Jul 21 21:26:21.783535 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.783411    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5" event={"ID":"689cbd30-6f15-4603-b386-a8c7fa0a211b","Type":"ContainerDied","Data":"9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb"}
Jul 21 21:26:21.783535 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.783470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5" event={"ID":"689cbd30-6f15-4603-b386-a8c7fa0a211b","Type":"ContainerDied","Data":"5b63f29c4b80ba4570557c9b17c41b1ccae454ab2a064fd8c85491c7a7465dcf"}
Jul 21 21:26:21.783535 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.783493    2567 scope.go:117] "RemoveContainer" containerID="9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb"
Jul 21 21:26:21.793042 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.793021    2567 scope.go:117] "RemoveContainer" containerID="9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb"
Jul 21 21:26:21.793297 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:26:21.793278    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb\": container with ID starting with 9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb not found: ID does not exist" containerID="9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb"
Jul 21 21:26:21.793367 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.793303    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb"} err="failed to get container status \"9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb\": rpc error: code = NotFound desc = could not find container \"9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb\": container with ID starting with 9da5425aac71d6af2a15f6d1a09914146141d3a3c11c4d68f823928be15302fb not found: ID does not exist"
Jul 21 21:26:21.806628 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.806607    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"]
Jul 21 21:26:21.810886 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:21.810864    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-0c5ebd-654977c584-kk4m5"]
Jul 21 21:26:23.312069 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:23.312038    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="689cbd30-6f15-4603-b386-a8c7fa0a211b" path="/var/lib/kubelet/pods/689cbd30-6f15-4603-b386-a8c7fa0a211b/volumes"
Jul 21 21:26:25.617272 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:25.617188    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:26.200471 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:26.200435    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:31.883120 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:31.883075    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:31.988912 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:31.988877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:32.188675 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:32.188641    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:32.431899 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:32.431856    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:33.246739 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:33.246704    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:33.352213 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:33.352168    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:42.973867 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:42.973830    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"]
Jul 21 21:26:42.974525 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:42.974178    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx" podUID="bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" containerName="maas-api" containerID="cri-o://2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239" gracePeriod=30
Jul 21 21:26:43.003094 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.003061    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:43.227733 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.227674    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:26:43.318249 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.318221    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnc7x\" (UniqueName: \"kubernetes.io/projected/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-kube-api-access-qnc7x\") pod \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") "
Jul 21 21:26:43.318424 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.318254    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls\") pod \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") "
Jul 21 21:26:43.318424 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.318302    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-openshift-service-ca\") pod \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\" (UID: \"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc\") "
Jul 21 21:26:43.318758 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.318716    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" (UID: "bec2a8ad-4c2e-401e-8e01-900bd95cb5cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:26:43.320349 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.320323    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" (UID: "bec2a8ad-4c2e-401e-8e01-900bd95cb5cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 21:26:43.320349 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.320333    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-kube-api-access-qnc7x" (OuterVolumeSpecName: "kube-api-access-qnc7x") pod "bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" (UID: "bec2a8ad-4c2e-401e-8e01-900bd95cb5cc"). InnerVolumeSpecName "kube-api-access-qnc7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:26:43.419084 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.419048    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnc7x\" (UniqueName: \"kubernetes.io/projected/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-kube-api-access-qnc7x\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:43.419084 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.419080    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-maas-api-tls\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:43.419084 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.419089    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc-openshift-service-ca\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:43.900700 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.900669    2567 generic.go:356] "Generic (PLEG): container finished" podID="bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" containerID="2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239" exitCode=0
Jul 21 21:26:43.900700 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.900706    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx" event={"ID":"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc","Type":"ContainerDied","Data":"2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239"}
Jul 21 21:26:43.901012 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.900713    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"
Jul 21 21:26:43.901012 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.900742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx" event={"ID":"bec2a8ad-4c2e-401e-8e01-900bd95cb5cc","Type":"ContainerDied","Data":"ec0b5f79dcf22fddc7974479c19abe6631e8e5c535e14b877891bba2a3d530bd"}
Jul 21 21:26:43.901012 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.900764    2567 scope.go:117] "RemoveContainer" containerID="2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239"
Jul 21 21:26:43.909910 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.909891    2567 scope.go:117] "RemoveContainer" containerID="2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239"
Jul 21 21:26:43.910172 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:26:43.910151    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239\": container with ID starting with 2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239 not found: ID does not exist" containerID="2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239"
Jul 21 21:26:43.910230 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.910180    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239"} err="failed to get container status \"2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239\": rpc error: code = NotFound desc = could not find container \"2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239\": container with ID starting with 2c66b76af744ee5b9ff3405e26efdaff7fc634163b66e6620aa7a4a60a171239 not found: ID does not exist"
Jul 21 21:26:43.922215 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.922193    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"]
Jul 21 21:26:43.926449 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:43.926427    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-80e9ec-589bfcd944-gqqlx"]
Jul 21 21:26:45.310869 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:45.310831    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" path="/var/lib/kubelet/pods/bec2a8ad-4c2e-401e-8e01-900bd95cb5cc/volumes"
Jul 21 21:26:47.635460 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:47.635423    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:48.067058 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:48.067027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:54.150817 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:54.150767    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:54.214638 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:54.214604    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:54.424092 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:54.424060    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"]
Jul 21 21:26:54.424332 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:54.424307    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy" containerID="cri-o://973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c" gracePeriod=30
Jul 21 21:26:54.458755 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:54.458719    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:55.005654 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:55.005617    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:26:59.682344 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.682320    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:26:59.766263 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766179    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-socket\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766263 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766234    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b42f7eb1-15cf-46e6-89c4-1abe81167120-istiod-ca-cert\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766502 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766275    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-credential-socket\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766502 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766290    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-certs\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766502 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766314    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-podinfo\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766502 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766330    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-envoy\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766502 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766345    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sq4pn\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-kube-api-access-sq4pn\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766502 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766399    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-data\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766502 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766442    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-token\") pod \"b42f7eb1-15cf-46e6-89c4-1abe81167120\" (UID: \"b42f7eb1-15cf-46e6-89c4-1abe81167120\") "
Jul 21 21:26:59.766863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766510    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:26:59.766863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766549    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:26:59.766863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766567    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:26:59.766863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766734    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b42f7eb1-15cf-46e6-89c4-1abe81167120-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 21:26:59.766863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766786    2567 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-socket\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.766863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766807    2567 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-credential-socket\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.766863 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.766823    2567 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-workload-certs\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.767323 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.767302    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-data" (OuterVolumeSpecName: "istio-data") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:26:59.768570 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.768545    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-kube-api-access-sq4pn" (OuterVolumeSpecName: "kube-api-access-sq4pn") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "kube-api-access-sq4pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:26:59.768710 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.768623    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 21:26:59.768998 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.768961    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 21:26:59.769077 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.768975    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-token" (OuterVolumeSpecName: "istio-token") pod "b42f7eb1-15cf-46e6-89c4-1abe81167120" (UID: "b42f7eb1-15cf-46e6-89c4-1abe81167120"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 21:26:59.867264 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.867219    2567 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-data\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.867264 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.867258    2567 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-token\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.867264 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.867268    2567 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b42f7eb1-15cf-46e6-89c4-1abe81167120-istiod-ca-cert\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.867264 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.867283    2567 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-podinfo\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.867523 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.867292    2567 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b42f7eb1-15cf-46e6-89c4-1abe81167120-istio-envoy\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.867523 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.867301    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq4pn\" (UniqueName: \"kubernetes.io/projected/b42f7eb1-15cf-46e6-89c4-1abe81167120-kube-api-access-sq4pn\") on node \"ip-10-0-130-177.ec2.internal\" DevicePath \"\""
Jul 21 21:26:59.993122 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.993083    2567 generic.go:356] "Generic (PLEG): container finished" podID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerID="973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c" exitCode=0
Jul 21 21:26:59.993282 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.993134    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"
Jul 21 21:26:59.993282 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.993140    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" event={"ID":"b42f7eb1-15cf-46e6-89c4-1abe81167120","Type":"ContainerDied","Data":"973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c"}
Jul 21 21:26:59.993282 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.993193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt" event={"ID":"b42f7eb1-15cf-46e6-89c4-1abe81167120","Type":"ContainerDied","Data":"a59a0ffac207e1414e07d659f63a51572d09cb252659c3eaf05d238a1768581c"}
Jul 21 21:26:59.993282 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:26:59.993224    2567 scope.go:117] "RemoveContainer" containerID="973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c"
Jul 21 21:27:00.002466 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:00.002444    2567 scope.go:117] "RemoveContainer" containerID="a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790"
Jul 21 21:27:00.010458 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:00.010437    2567 scope.go:117] "RemoveContainer" containerID="973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c"
Jul 21 21:27:00.010697 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:27:00.010676    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c\": container with ID starting with 973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c not found: ID does not exist" containerID="973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c"
Jul 21 21:27:00.010775 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:00.010710    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c"} err="failed to get container status \"973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c\": rpc error: code = NotFound desc = could not find container \"973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c\": container with ID starting with 973e03c21009a595eb96fb868887cd7f87fc571e10c0f20d9ae42a73b043c99c not found: ID does not exist"
Jul 21 21:27:00.010775 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:00.010733    2567 scope.go:117] "RemoveContainer" containerID="a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790"
Jul 21 21:27:00.011098 ip-10-0-130-177 kubenswrapper[2567]: E0721 21:27:00.010979    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790\": container with ID starting with a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790 not found: ID does not exist" containerID="a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790"
Jul 21 21:27:00.011145 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:00.011106    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790"} err="failed to get container status \"a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790\": rpc error: code = NotFound desc = could not find container \"a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790\": container with ID starting with a9fdfd8699e4e5a0c2f7d7f04f960a5beefe677aaa23a8f5ceb03f2b2baa8790 not found: ID does not exist"
Jul 21 21:27:00.016601 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:00.016544    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"]
Jul 21 21:27:00.019557 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:00.019536    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-80e9ec-openshift-default-78bc5759dc-2hstt"]
Jul 21 21:27:01.310816 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:01.310780    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" path="/var/lib/kubelet/pods/b42f7eb1-15cf-46e6-89c4-1abe81167120/volumes"
Jul 21 21:27:01.323919 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:01.323880    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-j7w99_b84d8279-e5c7-46fd-b42c-f6e52922ed23/manager/0.log"
Jul 21 21:27:01.453612 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:01.453587    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84f9cc6f45-bfwsb_1d31704d-555a-4831-bd20-bc2afd77f6ef/manager/0.log"
Jul 21 21:27:02.101750 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:02.101719    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647bdf6675-bsdv5_2b50bada-ba17-49dd-98c3-3f1a0b3d6255/copy-manifests/0.log"
Jul 21 21:27:02.108150 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:02.108127    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647bdf6675-bsdv5_2b50bada-ba17-49dd-98c3-3f1a0b3d6255/manager/0.log"
Jul 21 21:27:02.483325 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:02.483298    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6f45cb59b7-pmg28_49b9d3a7-de1e-4fc0-9e33-165e22403414/maas-api/0.log"
Jul 21 21:27:02.623443 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:02.623409    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744460-xg7hn_2baddef3-e6a1-4567-aaa5-72664ef16034/cleanup/0.log"
Jul 21 21:27:02.870169 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:02.870080    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mkhxt_48adffdd-ae68-4b19-bccb-351ce95b13ac/postgres/0.log"
Jul 21 21:27:03.738769 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:03.738745    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh_a7d9fdeb-f64c-4b40-a579-5c588fc20786/util/0.log"
Jul 21 21:27:03.748094 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:03.748070    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh_a7d9fdeb-f64c-4b40-a579-5c588fc20786/pull/0.log"
Jul 21 21:27:03.759466 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:03.759442    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh_a7d9fdeb-f64c-4b40-a579-5c588fc20786/extract/0.log"
Jul 21 21:27:03.881371 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:03.881344    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n_4599dfc0-81db-4dee-9d05-0e6f4b91bdb6/util/0.log"
Jul 21 21:27:03.888295 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:03.888271    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n_4599dfc0-81db-4dee-9d05-0e6f4b91bdb6/pull/0.log"
Jul 21 21:27:03.895218 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:03.895197    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n_4599dfc0-81db-4dee-9d05-0e6f4b91bdb6/extract/0.log"
Jul 21 21:27:04.023497 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:04.023417    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj_4e842868-4f6c-4dbb-882c-e913c9f7b20b/pull/0.log"
Jul 21 21:27:04.031483 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:04.031458    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj_4e842868-4f6c-4dbb-882c-e913c9f7b20b/extract/0.log"
Jul 21 21:27:04.041278 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:04.041256    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj_4e842868-4f6c-4dbb-882c-e913c9f7b20b/util/0.log"
Jul 21 21:27:04.164972 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:04.164938    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm_a024eea2-3868-406d-aadd-30673217b8f1/util/0.log"
Jul 21 21:27:04.170969 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:04.170945    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm_a024eea2-3868-406d-aadd-30673217b8f1/pull/0.log"
Jul 21 21:27:04.177295 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:04.177249    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm_a024eea2-3868-406d-aadd-30673217b8f1/extract/0.log"
Jul 21 21:27:04.554675 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:04.554644    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-w8ns6_97d6b751-1106-4842-9122-bb1b5bec784d/manager/0.log"
Jul 21 21:27:04.945130 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:04.945102    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hnp24_2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65/manager/0.log"
Jul 21 21:27:05.695755 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:05.695723    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pxkdh_66657f1f-bee9-4f60-bfc8-231a7e196eee/discovery/0.log"
Jul 21 21:27:05.932108 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:05.932077    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65495b7cc9-8w4z4_4f5f8352-40f8-4213-ba2f-f1e4963e92e2/kube-auth-proxy/0.log"
Jul 21 21:27:06.053276 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:06.053199    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bnqdr_55009324-0e72-44b6-a2df-5652f7aaaaab/istio-proxy/1.log"
Jul 21 21:27:06.184828 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:06.184791    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-b48b6_58bae39a-f148-4e65-b2ae-80093e13b586/payload-pre-processing/0.log"
Jul 21 21:27:06.809605 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:06.809576    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-99wth_b9d08045-fb6e-43bf-ac85-62f9b109f2f9/main/0.log"
Jul 21 21:27:07.062758 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:07.062683    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pjd7n_7bad3452-89ad-4d26-a105-22b7fdf4f4d8/main/0.log"
Jul 21 21:27:07.182292 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:07.182263    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f66zkx_b43d70eb-ece3-4b8a-b524-3fc7e0ad2af7/main/0.log"
Jul 21 21:27:12.448282 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:12.448247    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:27:12.549696 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:12.549654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:27:13.746456 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:13.746417    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:27:13.807555 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:13.807520    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log" limit=1024
Jul 21 21:27:16.621790 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:16.621759    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pdlgq_cf604c6d-39e1-43a4-8cfc-9f06bf3cd729/global-pull-secret-syncer/0.log"
Jul 21 21:27:16.710513 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:16.710480    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-467q6_37291fdd-4b40-4d1a-af18-99d28c690348/konnectivity-agent/0.log"
Jul 21 21:27:16.829649 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:16.829618    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-177.ec2.internal_1399074c7686ed18dafa282fd37a44d0/haproxy/0.log"
Jul 21 21:27:20.446190 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.446136    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh_a7d9fdeb-f64c-4b40-a579-5c588fc20786/extract/0.log"
Jul 21 21:27:20.469170 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.469146    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh_a7d9fdeb-f64c-4b40-a579-5c588fc20786/util/0.log"
Jul 21 21:27:20.502840 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.502811    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fdtlh_a7d9fdeb-f64c-4b40-a579-5c588fc20786/pull/0.log"
Jul 21 21:27:20.540710 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.540683    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n_4599dfc0-81db-4dee-9d05-0e6f4b91bdb6/extract/0.log"
Jul 21 21:27:20.563492 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.563467    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n_4599dfc0-81db-4dee-9d05-0e6f4b91bdb6/util/0.log"
Jul 21 21:27:20.591365 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.591339    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rwk6n_4599dfc0-81db-4dee-9d05-0e6f4b91bdb6/pull/0.log"
Jul 21 21:27:20.633436 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.633411    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj_4e842868-4f6c-4dbb-882c-e913c9f7b20b/extract/0.log"
Jul 21 21:27:20.657480 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.657459    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj_4e842868-4f6c-4dbb-882c-e913c9f7b20b/util/0.log"
Jul 21 21:27:20.680932 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.680905    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed739l6pj_4e842868-4f6c-4dbb-882c-e913c9f7b20b/pull/0.log"
Jul 21 21:27:20.726275 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.726208    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm_a024eea2-3868-406d-aadd-30673217b8f1/extract/0.log"
Jul 21 21:27:20.746944 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.746912    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm_a024eea2-3868-406d-aadd-30673217b8f1/util/0.log"
Jul 21 21:27:20.769721 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:20.769700    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tsvhm_a024eea2-3868-406d-aadd-30673217b8f1/pull/0.log"
Jul 21 21:27:21.183203 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:21.183174    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-w8ns6_97d6b751-1106-4842-9122-bb1b5bec784d/manager/0.log"
Jul 21 21:27:21.412136 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:21.412104    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-hnp24_2ee2bc77-b69b-4fd7-b8b0-e2e3e1a9be65/manager/0.log"
Jul 21 21:27:23.404352 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:23.404317    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-plspl_2bf8996a-81f7-490c-acc0-a201ff2a902e/cluster-monitoring-operator/0.log"
Jul 21 21:27:23.751612 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:23.751580    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rcb59_cbdbf8ca-3542-475f-b293-f66fd9eb2f79/node-exporter/0.log"
Jul 21 21:27:23.781893 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:23.781870    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rcb59_cbdbf8ca-3542-475f-b293-f66fd9eb2f79/kube-rbac-proxy/0.log"
Jul 21 21:27:23.803695 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:23.803670    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rcb59_cbdbf8ca-3542-475f-b293-f66fd9eb2f79/init-textfile/0.log"
Jul 21 21:27:27.500508 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500476    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"]
Jul 21 21:27:27.500895 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500834    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="689cbd30-6f15-4603-b386-a8c7fa0a211b" containerName="maas-api"
Jul 21 21:27:27.500895 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500846    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="689cbd30-6f15-4603-b386-a8c7fa0a211b" containerName="maas-api"
Jul 21 21:27:27.500895 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500857    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" containerName="maas-api"
Jul 21 21:27:27.500895 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500862    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" containerName="maas-api"
Jul 21 21:27:27.500895 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500884    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy"
Jul 21 21:27:27.500895 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500889    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy"
Jul 21 21:27:27.500895 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500898    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy"
Jul 21 21:27:27.501151 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500904    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy"
Jul 21 21:27:27.501151 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500954    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bec2a8ad-4c2e-401e-8e01-900bd95cb5cc" containerName="maas-api"
Jul 21 21:27:27.501151 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500963    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy"
Jul 21 21:27:27.501151 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500968    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b42f7eb1-15cf-46e6-89c4-1abe81167120" containerName="istio-proxy"
Jul 21 21:27:27.501151 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.500975    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="689cbd30-6f15-4603-b386-a8c7fa0a211b" containerName="maas-api"
Jul 21 21:27:27.504119 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.504103    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.506637 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.506614    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-75fnx\"/\"kube-root-ca.crt\""
Jul 21 21:27:27.506751 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.506654    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-75fnx\"/\"openshift-service-ca.crt\""
Jul 21 21:27:27.507505 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.507485    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-75fnx\"/\"default-dockercfg-6ncg2\""
Jul 21 21:27:27.512908 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.512883    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"]
Jul 21 21:27:27.609615 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.609580    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-podres\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.609795 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.609619    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q4kj\" (UniqueName: \"kubernetes.io/projected/11b2ff4a-ead8-479a-9eb6-7b332a612adf-kube-api-access-7q4kj\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.609795 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.609654    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-proc\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.609795 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.609674    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-lib-modules\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.609795 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.609718    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-sys\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.710966 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.710922    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-podres\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.710966 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.710970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q4kj\" (UniqueName: \"kubernetes.io/projected/11b2ff4a-ead8-479a-9eb6-7b332a612adf-kube-api-access-7q4kj\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.711303 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.711025    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-proc\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.711303 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.711047    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-lib-modules\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.711303 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.711092    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-sys\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.711303 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.711117    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-podres\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.711303 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.711185    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-sys\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.711303 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.711124    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-proc\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.711303 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.711225    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11b2ff4a-ead8-479a-9eb6-7b332a612adf-lib-modules\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.719772 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.719753    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q4kj\" (UniqueName: \"kubernetes.io/projected/11b2ff4a-ead8-479a-9eb6-7b332a612adf-kube-api-access-7q4kj\") pod \"perf-node-gather-daemonset-ssm94\" (UID: \"11b2ff4a-ead8-479a-9eb6-7b332a612adf\") " pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.814708 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.814620    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:27.948490 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:27.948397    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"]
Jul 21 21:27:27.950764 ip-10-0-130-177 kubenswrapper[2567]: W0721 21:27:27.950739    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod11b2ff4a_ead8_479a_9eb6_7b332a612adf.slice/crio-a796d77ae2350692518adc2b087e520c6976c4792924bedd8727d4ac35a11b7f WatchSource:0}: Error finding container a796d77ae2350692518adc2b087e520c6976c4792924bedd8727d4ac35a11b7f: Status 404 returned error can't find the container with id a796d77ae2350692518adc2b087e520c6976c4792924bedd8727d4ac35a11b7f
Jul 21 21:27:28.141561 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:28.141464    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94" event={"ID":"11b2ff4a-ead8-479a-9eb6-7b332a612adf","Type":"ContainerStarted","Data":"f8fbd854f14456ed6fbdc9d3b6ee14a12ded855d5b6fa731318439a75dbff2ba"}
Jul 21 21:27:28.141561 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:28.141531    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:28.141561 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:28.141554    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94" event={"ID":"11b2ff4a-ead8-479a-9eb6-7b332a612adf","Type":"ContainerStarted","Data":"a796d77ae2350692518adc2b087e520c6976c4792924bedd8727d4ac35a11b7f"}
Jul 21 21:27:28.156860 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:28.156811    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94" podStartSLOduration=1.156794641 podStartE2EDuration="1.156794641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 21:27:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 21:27:28.156287611 +0000 UTC m=+2875.366316070" watchObservedRunningTime="2026-07-21 21:27:28.156794641 +0000 UTC m=+2875.366823083"
Jul 21 21:27:28.994130 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:28.994097    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7qx6q_2879e13a-d9ac-4b68-8353-c8f2d717423f/dns/0.log"
Jul 21 21:27:29.015532 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:29.015508    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7qx6q_2879e13a-d9ac-4b68-8353-c8f2d717423f/kube-rbac-proxy/0.log"
Jul 21 21:27:29.136664 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:29.136630    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n2h4z_a56b897c-e624-4d27-a9bb-8e36b5121ce2/dns-node-resolver/0.log"
Jul 21 21:27:29.765800 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:29.765772    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lcbgb_33b3e3b1-3e9e-4ed9-9aa4-68038f31d9fb/node-ca/0.log"
Jul 21 21:27:30.847732 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:30.847696    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pxkdh_66657f1f-bee9-4f60-bfc8-231a7e196eee/discovery/0.log"
Jul 21 21:27:30.894316 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:30.894286    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65495b7cc9-8w4z4_4f5f8352-40f8-4213-ba2f-f1e4963e92e2/kube-auth-proxy/0.log"
Jul 21 21:27:30.923156 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:30.923125    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bnqdr_55009324-0e72-44b6-a2df-5652f7aaaaab/istio-proxy/0.log"
Jul 21 21:27:30.950185 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:30.950159    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bnqdr_55009324-0e72-44b6-a2df-5652f7aaaaab/istio-proxy/1.log"
Jul 21 21:27:30.983718 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:30.983694    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-b48b6_58bae39a-f148-4e65-b2ae-80093e13b586/payload-pre-processing/0.log"
Jul 21 21:27:31.775529 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:31.775496    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p5j5p_820520d0-8efa-4215-bbe9-30746248ce48/serve-healthcheck-canary/0.log"
Jul 21 21:27:32.308049 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:32.308022    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-lk8hn_5503f039-d91f-464c-8731-8dbbecc97c04/insights-operator/0.log"
Jul 21 21:27:32.308755 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:32.308740    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-lk8hn_5503f039-d91f-464c-8731-8dbbecc97c04/insights-operator/1.log"
Jul 21 21:27:32.328868 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:32.328841    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bvwwn_37d8148e-f700-4a83-924f-a60ce06386e9/kube-rbac-proxy/0.log"
Jul 21 21:27:32.348818 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:32.348801    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bvwwn_37d8148e-f700-4a83-924f-a60ce06386e9/exporter/0.log"
Jul 21 21:27:32.372914 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:32.372896    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bvwwn_37d8148e-f700-4a83-924f-a60ce06386e9/extractor/0.log"
Jul 21 21:27:34.155847 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:34.155773    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-75fnx/perf-node-gather-daemonset-ssm94"
Jul 21 21:27:34.806255 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:34.806221    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-j7w99_b84d8279-e5c7-46fd-b42c-f6e52922ed23/manager/0.log"
Jul 21 21:27:34.912440 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:34.912413    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-84f9cc6f45-bfwsb_1d31704d-555a-4831-bd20-bc2afd77f6ef/manager/0.log"
Jul 21 21:27:35.182025 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:35.181975    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647bdf6675-bsdv5_2b50bada-ba17-49dd-98c3-3f1a0b3d6255/manager/0.log"
Jul 21 21:27:35.203708 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:35.203686    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647bdf6675-bsdv5_2b50bada-ba17-49dd-98c3-3f1a0b3d6255/copy-manifests/0.log"
Jul 21 21:27:37.307495 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:37.307460    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pxkdh_66657f1f-bee9-4f60-bfc8-231a7e196eee/discovery/0.log"
Jul 21 21:27:37.336626 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:37.336594    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65495b7cc9-8w4z4_4f5f8352-40f8-4213-ba2f-f1e4963e92e2/kube-auth-proxy/0.log"
Jul 21 21:27:37.364240 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:37.364217    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bnqdr_55009324-0e72-44b6-a2df-5652f7aaaaab/istio-proxy/0.log"
Jul 21 21:27:37.389349 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:37.389323    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bnqdr_55009324-0e72-44b6-a2df-5652f7aaaaab/istio-proxy/1.log"
Jul 21 21:27:37.411254 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:37.411226    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-b48b6_58bae39a-f148-4e65-b2ae-80093e13b586/payload-pre-processing/0.log"
Jul 21 21:27:38.766671 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:38.766570    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pxkdh_66657f1f-bee9-4f60-bfc8-231a7e196eee/discovery/0.log"
Jul 21 21:27:38.798729 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:38.798690    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65495b7cc9-8w4z4_4f5f8352-40f8-4213-ba2f-f1e4963e92e2/kube-auth-proxy/0.log"
Jul 21 21:27:38.819540 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:38.819508    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bnqdr_55009324-0e72-44b6-a2df-5652f7aaaaab/istio-proxy/0.log"
Jul 21 21:27:38.850957 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:38.850927    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-bnqdr_55009324-0e72-44b6-a2df-5652f7aaaaab/istio-proxy/1.log"
Jul 21 21:27:38.873522 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:38.873495    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-b48b6_58bae39a-f148-4e65-b2ae-80093e13b586/payload-pre-processing/0.log"
Jul 21 21:27:41.863838 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:41.863782    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-vrg42_772a7ff6-26b0-445a-a319-5eb76fd9b180/migrator/0.log"
Jul 21 21:27:41.884514 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:41.884487    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-vrg42_772a7ff6-26b0-445a-a319-5eb76fd9b180/graceful-termination/0.log"
Jul 21 21:27:43.230113 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.230084    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6f7nt_d8fd5a6d-ebe9-432c-b317-d1ae72fc149b/kube-multus/0.log"
Jul 21 21:27:43.562509 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.562487    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc6xm_d3004918-748e-45ab-85e8-7eefa1d359b3/kube-multus-additional-cni-plugins/0.log"
Jul 21 21:27:43.583608 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.583574    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc6xm_d3004918-748e-45ab-85e8-7eefa1d359b3/egress-router-binary-copy/0.log"
Jul 21 21:27:43.604635 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.604613    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc6xm_d3004918-748e-45ab-85e8-7eefa1d359b3/cni-plugins/0.log"
Jul 21 21:27:43.624916 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.624893    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc6xm_d3004918-748e-45ab-85e8-7eefa1d359b3/bond-cni-plugin/0.log"
Jul 21 21:27:43.645810 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.645789    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc6xm_d3004918-748e-45ab-85e8-7eefa1d359b3/routeoverride-cni/0.log"
Jul 21 21:27:43.666427 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.666401    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc6xm_d3004918-748e-45ab-85e8-7eefa1d359b3/whereabouts-cni-bincopy/0.log"
Jul 21 21:27:43.687194 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.687169    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wc6xm_d3004918-748e-45ab-85e8-7eefa1d359b3/whereabouts-cni/0.log"
Jul 21 21:27:43.863917 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.863838    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tms9z_cf884bc1-53a5-47ab-98cd-3959ded9e6e7/network-metrics-daemon/0.log"
Jul 21 21:27:43.882553 ip-10-0-130-177 kubenswrapper[2567]: I0721 21:27:43.882530    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tms9z_cf884bc1-53a5-47ab-98cd-3959ded9e6e7/kube-rbac-proxy/0.log"